Recruiting · Production

JUDE: LinkedIn's LLM-based embedding platform for job recommendations

The problem

Deploying LLMs in production at LinkedIn's scale brought high computational costs, complex deployment pipelines, and continuous domain adaptation needs that the previous embedding platform Pensieve could not address efficiently.

First attempt

The previous embedding platform Pensieve relied on imprecise smaller ML models, hard-to-maintain taxonomies, and rigid upstream pipelines using Lambda architecture, which caused time-travel data consistency issues and required monitoring and recovery of failed scheduled inference jobs.

Workflow diagram · grounded in source

Entity change triggers pipeline

trigger

“We have separate input Kafka / Brooklin streams representing the changelog for each entity, which trigger embedding inference for their respective entities.”

LLM fine-tuning with dual supervision

ai_action

“In JUDE, we use relevance labels during initial training phases to establish baseline semantic understanding, then incorporate engagement labels to align with business outcomes. This dual-signal approach enables our embeddings to simulta…”

Change detection skips unchanged inputs

validation

“Change detection: Skip inference if the text content has not changed meaningfully from the previous version. This simple optimization reduces the embedding inference cost up to 3x.”

Nearline LLM embedding inference

ai_action

“Embedding inference: Make a GRPC call to the LLM Model Serving cluster to generate the embedding for the text input. For a modern 7B LLM, latency remains under 300ms at p95 quantile”

Embeddings stored in Venice and HDFS

integration

“Online storage: Embeddings are stored in Venice, a high-performance key-value store for real-time access during document ranking. Offline storage: Generated embeddings are published into Kafka topics that are ETL'd to HDFS for use in mod…”

Job recommendations delivered via JUDE embeddings

output

“JUDE Embeddings have been ramped online, replacing the overlapping standardized features in both job recommendation and search L2 ranking models, leading to +2.07% Qualified Applications, -5.13% Dismiss to Apply, +1.91% Total Job Applica…”

Reported outcome

JUDE embeddings ramped online replaced standardized features in job recommendation and search L2 ranking models, delivering +2.07% Qualified Applications, -5.13% Dismiss to Apply, and +1.91% Total Job Applications — described as the highest metric improvement from a single model change the team had observed that half year.

Reported metrics

Qualified Applications+2.07%

Dismiss to Apply rate-5.13%

Total Job Applications+1.91%

Embedding inference cost reduction from change detectionup to 3x

Show all 7 reported metrics

Qualified Applications+2.07%

Dismiss to Apply rate-5.13%

Total Job Applications+1.91%

Embedding inference cost reduction from change detectionup to 3x

Inference volume reduction from change detectionapproximately 6x

Performance gap bridged by cross-encoder distillation50%

Embedding inference latency at p95under 300ms

Reported stack

PyTorchHugging FaceKafkaBrooklinSamzaVeniceHDFSModel CloudDeepSpeedLigerLoRAFlash attention 2FlyteSparkRayKubernetes

Source

https://www.linkedin.com/blog/engineering/ai/jude-llm-based-representation-learning-for-linkedin-job-recommendations

Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

JUDE embeddings ramped online replaced standardized features in job recommendation and search L2 ranking models, delivering +2.07% Qualified Applications, -5.13% Dismiss to Apply, and +1.91% Total Job Applications — d…

What tools did this team use?

PyTorch, Hugging Face, Kafka, Brooklin, Samza, Venice, HDFS, Model Cloud, DeepSpeed, Liger.

What results were reported?

Qualified Applications: +2.07%; Dismiss to Apply rate: -5.13%; Total Job Applications: +1.91%; Embedding inference cost reduction from change detection: up to 3x (source-reported, not independently verified).

What failed first in this deployment?

How is this recruiting AI workflow structured?

Entity change triggers pipeline → LLM fine-tuning with dual supervision → Change detection skips unchanged inputs → Nearline LLM embedding inference → Embeddings stored in Venice and HDFS → Job recommendations delivered via JUDE embeddings.