Data eXplore : Data Science, ML, Big Data, LLMs and AI Security

Gradient Stability in Long Sequences How Spherical Gradient Protects Against Explosions and Vanishing Gradients in Online Time Series Learning? In online time series learning, gradients either explode or vanish when dealing with long sequences. LSTMs and GRUs handle this instability poorly: with streaming data, the model fails to adapt to new patterns due to the exponential growth or collapse of the gradient. The …

Delayed Feedback In CVR Models How to avoid breaking conversion learning and evaluation due to delayed labels? In CVR, a user may convert minutes, days, or weeks after clicking, so labeling "as is" often turns future positive conversions into false negatives. This is critical for advertising, recommendations, marketplace funnels, and A/B tests, where the model is trained on incomplete logs. Problem: label may not …

Adding reactions to posts is disabled from now on. We are also reviewing the save and forward options.

Updated the encoder - broke the ANN? How to migrate embeddings without pain In embedding-based systems, the encoder is part of the data contract. It cannot be updated like a regular ML model: the ANN index already contains vectors from the old space, and a common mistake is to assume compatibility due to the same dimension and metric. Why compatibility breaks? Even if the dimension is the same and the cosine is th…

How to Find Harmful Training Examples Before Fine-Tuning: Influence Functions, TracIn, and Data Pruning in Production ML In production ML, "bad" training examples can be costly: a cluster of mislabeled, outdated, or anomalous objects can consistently degrade fine-tuning on a fresh data set. A common mistake is to clean the dataset only based on heuristics and not checking which samples actually increase the loss on …

Temporal leakage in the feature store: How point-in-time joins, backfills, and feature causality checks save a model from beautiful offline metrics and failure in production? Temporal leakage in the feature store is one of the most expensive ways to get great offline metrics and a useless model in production. The problem isn't that the feature is bad, but that on train it knows more than the model would have known a…

Conformal intervals in production ML with covariate shift: How to maintain coverage without unnecessarily wide predictions? Split conformal works well with exchangeability: train, calibration, and test come from same distribution. In production, this often breaks down due to geo, devices, channels, seasonality or a change in acquisition mix and a common mistake is to simply expand intervals "with a margin". What ex…

⁣Corpus drift in RAG systems: How to notice the degradation of retrieval without labels, annotations, and obvious errors? In RAG retrieval, things often break silently: same model, same embedding model, same prompt, normal latency, but the answers have gotten worse. A typical mistake is to immediately tweak the prompt or blame the LLM, even though the problem lies deeper: the corpus has changed. 1️⃣ Monitor the co…

ML process template that we use in the ML Core team Maybe the template will make it easier for you to start putting together a doc: 1️⃣ Create a scheme of the main development stages adopted in your team, for example: • task setting; • data exploration; • formulating the task in ML terms; • MVP solution; • testing the solution; • rolling it out to production; • monitoring. 2️⃣ Describe each stage: • what needs to …

In the previous post, i explained what conformal prediction is and why it's needed. But you probably have some questions: 1. How does the model understand how "strange" or "risky" a particular object is? 2. How is the conformal predictor trained? The answer to the first question: through the measure of discomfort. The measure of discomfort is a function that shows how poorly a particular pair (x, y) matches the mod…

EVOLUTION OF ATTENTION: Transition to Linear Models This is a continuation of the series of posts about the path of linear Attention. Last time, we found out that transformers have quadratic complexity, which makes them poorly scalable for long sequences and requires a lot of memory. ➡️ How it works? 📌 Linear Attention first changed the approach to computation. It allowed to decompose the kernel function and inste…

EVOLUTION OF ATTENTION: From RNN to Transformer Starting a series of posts to explain path to linear Attention. Before transformers, tasks like translation and classification mainly used recurrent models (RNNs). In 2014, the Attention mechanism appeared. It allowed us not just to read a text sequentially, but to look at all input tokens and assess which of them were important for generation. ➡️ How it worked? A bi…