Practice Questions with Worked Solutions

And the five undercurrents, which run underneath every stage rather than between them: security, data management, DataOps, data architecture, orchestration.

Remember which are undercurrents by asking: is this true of ingestion AND of serving? Security is. Orchestration is. That is why they are drawn underneath.

A crash is loud and gets fixed. A wrong number is trusted.

01_environment_etl.py runs an ETL job on data with three defects — a lowercase region, a price with a currency prefix, a duplicated order:

The three defects were worth −₹1,680, −₹700 and +₹2,520 and nearly cancelled.

That is the dangerous case. A figure that is wildly wrong gets noticed; a figure 1% wrong gets reported to the board. The errors cancelled by luck.

The only reliable check is an independently computed figure, which is why this repository has five engines agreeing on ₹10,360.

ELT won because cloud warehouse compute became cheap and elastic. Keeping the raw data means a transformation bug is fixed by re-running the transformation, rather than re-extracting from a source system that may no longer hold the history.

Each nine costs roughly ten times the last, so ask what downtime actually costs before choosing.

RPO is the one people forget. A system that recovers in five minutes but loses an hour of transactions has an excellent RTO and an unacceptable RPO.

Start by conceding the point. 01_environment_etl.py times both: every boundary adds serialisation, a network hop and a new failure mode, so the microservices version is measurably slower.

What you buy is not speed. It is independent deployment and independent scaling, and they only pay when the parts genuinely differ:

That is an argument about cost at scale, not elegance.

And the hidden cost: one database per service. If two services share a database they are not independent, so the data must be split too — and a join you used to write in SQL becomes an API call and an in-memory merge.

The rule: start with a well-structured monolith; split a service out when you can name the specific scaling or deployment problem it solves.

04_batch_vs_event.py, 300 records, 2 ms work each, 250 ms batch interval:

About 160×, because a batch record waits for the next tick — its latency is dominated by the interval, not the 2 ms of processing.

Batching does not make you faster. It makes you efficient per record at the cost of latency per record.

The failure row decides real architectures. Re-running a whole batch requires every step to be idempotent, which is most of the work in writing an ETL job. A stream consumer retries one message but needs a dead-letter queue for one that fails for ever.

"Free" open-source software with a full-time engineer maintaining it costs more than a managed service at ₹40,000 a month.

And the related distinction: CapEx (buy servers) against OpEx (rent them). The move to cloud shifts one to the other, changing the accounting as much as the engineering.

The rollback row causes real incidents. Deploying an old model with new preprocessing produces predictions that are silently wrong — no error, no crash, just worse decisions.

The transformation applied at training differs from the one applied at serving.

Example: you train on a DataFrame where a missing value became the column mean; you serve from a Flask endpoint where a missing value becomes zero. The model is fine; the input is not, and nothing raises an error.

The fix is architectural, not a test: one implementation of the transformation, called from both paths. That is what a feature store is for, and a shared module is enough at small scale.

Number 4 is the one people forget. 07_mlflow_dvc.py demonstrates it: the same code with the model still seeded and only train_test_split's random_state removed, run three times, gives three different AUCs.

That is enough to make every number in a report unrepeatable.

The pinned version runs twice and gives byte-identical results — 0.793326 both times.

Git tracks the pointer; DVC keeps the data.

The .dvc file is a few hundred bytes holding an md5 and a size. The data file itself is in .gitignore and lives in the DVC cache or a remote.

The two-command workflow:

git checkout <sha>     # moves the code AND the pointer
dvc checkout           # reads the pointer, restores the data

07_mlflow_dvc.py verifies this: committing a drifted v2, checking out v1, and confirming the income mean returns from 1.4805 to −0.0195.

Why it matters: when a production model behaves oddly, the question is "what did it see when it was trained". Without data versioning that question has no answer.

Because the gap is the only column that says why a model underperformed.

From 07_mlflow_dvc.py:

rf-depth12 scored worst and has a gap of 0.2783 — it memorised the training set. Without the gap column you know only that it was worse, not that the fix is regularisation rather than more features.

It costs one extra log_metric call.

Handling the requests that are wrong.

12_serve_drift_govern.py runs a real Flask server and calls it:

Returning 400 with a reason — rather than a 500, or a confidently wrong prediction on garbage input — is the difference. The model itself is three lines of that file.

Also required: a health endpoint, a version string in the response, structured logging, and a latency budget.

A CI step that fails the build when the trained model is not good enough.

if metrics["auc"] < MIN_AUC:
    sys.exit(f"AUC {metrics['auc']:.4f} below threshold")

This is what makes it an ML pipeline rather than a software one. Ordinary CI asks "does the code work?". ML CI must also ask "is the model good enough?" — a pipeline can run perfectly and produce a model worse than the one in production.

Compare against the current production model, not a fixed number.

And the prerequisite: the training must be deterministic, or the metric moves between runs and the gate becomes a coin toss. That is exactly what 07_mlflow_dvc.py verifies.

Shadow is the one to know beyond the syllabus. Running the new model on real production traffic while serving the old one's answer is the only way to see production behaviour at zero user risk — you find the training/serving skew, the unexpected nulls and the latency problem before anybody is affected.

The A/B trap: you are testing a business outcome, not an accuracy. A model with better AUC can produce worse revenue.

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py model.pkl ./

Docker caches each layer and invalidates every layer after the first change. Copying the code before installing means a one-line edit invalidates the pip install layer and reinstalls everything.

With this ordering, the expensive layer is rebuilt only when the dependencies change.

And the related trap: without a .dockerignore, COPY . . ships the whole .git directory and every cached dataset as build context — a 40 MB project becomes a 2 GB build.

Swapping them gives you a pod that is restarted every time it is briefly slow, which turns a latency spike into an outage — the restart drops in-flight requests and the replacement starts cold, making the latency worse.

Related: resources.requests is what the scheduler reserves; limits is where the container is killed. A container with no memory limit can take down its node, and an ML container loading a large model is exactly the kind that does.

The asymmetry that defines production monitoring: data drift is detectable immediately and often harmless; concept drift is damaging and cannot be detected without labels — which in lending arrive months later.

So you monitor what you can see as a proxy for what you cannot.

The drift shifted P(X) — incomes moved up. It did not shift P(y|X) — the generating coefficients were unchanged. A model that learned the true relationship is still correct on shifted inputs.

That is why 12_serve_drift_govern.py also reports accuracy on the drifted batches as slightly higher (0.7172) than on the clean ones (0.7060).

The operational rule: alert on data drift, INVESTIGATE, and retrain only if the labels confirm the relationship moved. Retraining on every input shift is expensive and can make things worse — you fit a smaller, more recent, noisier window.

Why PSI rather than KS: the KS test gives a p-value, and with enough samples any shift is statistically significant.

From 12_serve_drift_govern.py:

The KS p-value hits 0.0000 at batch 5, detecting a shift far too small to matter operationally.

Statistical significance is not operational significance. PSI measures a magnitude, so a threshold on it means something. This is the same point Statistical Foundations for Data Science makes about p-values and effect sizes.

Result: 4 of 5 drifted batches caught, 0 false alarms, one-batch lag — and every detector trades lag against false alarms.

Article 17 is the right to erasure.

Deleting the row from the database is easy. The model trained on that row still contains information derived from it, and there is no way to subtract one example from fitted coefficients.

The practical answer: retrain on a schedule from current data, so an erased record leaves the model within one retraining cycle.

Which means your retraining pipeline is a compliance control, not just an accuracy one. That is not obvious and it is exactly the kind of connection an examiner is looking for.

Related, Article 22: a decision with legal effect — a loan approval is one — requires a human review route and an explanation. That constrains model choice before you fit anything: logistic regression can tell an applicant which factors weighed against them; a deep ensemble cannot.

You cannot tell from this table, and saying so is the correct answer.

In 12_serve_drift_govern.py the regions were generated from the same distribution, so the spread of 0.0325 is sampling noise — and knowing that is only possible because the data was constructed.

On real data the same table would not tell you either. A spread could be:

1. sampling noise,
2. a genuine difference in the applicant pool, or
3. the model.

Deciding which is not a statistical question — it needs the domain.

And note that excluding the protected attribute does not prevent bias. Proxies remain: a postcode, an employer, a name. Removing region from the feature list removes your ability to measure the disparity, not the disparity.

A gauge holding "the mean latency" cannot be aggregated.

Averaging the means of four instances is not the mean — it is only correct if all four served identical request counts. And p95 cannot be recovered from a mean at all.

A histogram ships bucket counts, which sum correctly across instances, and histogram_quantile() computes the percentile over the summed buckets.

12_serve_drift_govern.py uses a gauge for simplicity and its output says gauge honestly — that is a simplification, not a recommendation.

It requires the condition to hold continuously for ten minutes before the alert fires.

Without it, one slow request pages somebody at 03:00.

Choosing it is the whole craft of alerting: too short and you train people to ignore alerts; too long and you find out from a customer.

Drift alerts use for: 1h — drift is a slow phenomenon and one noisy batch should not page anyone.