How to Build an AI Factory in India: A Practical Blueprint
TL;DR: This is how to build an AI factory for Indian conditions: in a sequence, not from a box. Size it from the decisions it must produce and rent GPU capacity before you own any. Keep training data and model weights inside India by design. Run every model through one shared pipeline, with a governance gate in front of production before the first model asks for it. Give a standing Centre of Excellence the job of keeping all of it consistent. This post walks through the five decisions, the operating model, a phased plan and the numbers that prove the factory is working.
Most Indian enterprises do not have a shortage of AI ideas. They have a shortage of production. A bank pilots a document-reading model, a state department tests a citizen chatbot, a manufacturer trials predictive maintenance on one line. Six months later, all three remain in the pilot stage, while the GPUs purchased for them sit idle for most of the day.
Our earlier explainer covered what an AI factory is and why the term has stuck. This post picks up where that one stopped. It is about the building. Which decisions matter, in what order, roughly what they cost, and how to tell whether you have built a factory or an expensive GPU cluster with a good name.
Indian conditions shape every part of it. Power is constrained, GPU lead times are long, regulators in banking, insurance and government have firm views on where data may sit, and the IndiaAI Mission has reset what people expect to pay for compute. A blueprint copied from a hyperscaler playbook written for Virginia or Dublin will not fit any of that.
Start from the output, not the hardware!
The most common mistake is to size the factory from a GPU quotation. Size it instead of what it has to produce in its first year.
Write down the cases that are real: the ones with a business owner, a data source you can name and an outcome someone will be measured on. For most organisations this is a short list.
A lender might have fraud scoring, KYC document extraction and a collections assistant. A power distribution company might have load forecasting and an outage classifier. A hospital group, for instance, might prioritise radiology triage and automated claims coding as its initial AI use cases. Three to five is normal. Twenty is a sign that nobody has prioritized.
For each use case, estimate three numbers:
- How often is the model called, per day.
- How fast must it answer. Milliseconds for a payment check, a second or two for a chatbot, hours for an overnight forecast.
- How often it must be retrained? Weekly for a fraud model that adversary’s probe, quarterly for a document classifier.
Those three numbers settle almost everything downstream: the split between training and inference capacity, whether you need high-bandwidth GPU interconnects at all, and whether a managed GPU service works out cheaper than owning hardware.
The output of this exercise is a one-page demand sheet. Every later decision in this post refers back to it.
Five decisions that shape an AI factory
With the demand sheet in hand, the build comes down to five decisions. Get them in this order and the rest is engineering.
Owning a cluster makes sense when utilisation will stay high and the workloads are steady, which usually means a large training programme or a regulator that insists on dedicated, physically isolated hardware. Everyone else should start by renting. In our experience the realistic utilisation figure for a first-year enterprise cluster is well under half, and an idle B200 or GB200 GPU is the most expensive thing you can leave switched on.
The third option, and the one most Indian enterprise settle into, is a split estate: a small, dedicated pod for regulated or latency-critical workloads, and rented capacity for training bursts and experiments. The demand sheet tells you the size of each half.
Two India-specific factors push the answer towards managed capacity. The first is power and cooling. An air-cooled server with eight NVIDIA H200 GPUs is rated at roughly 7.5 kW, and a liquid-cooled GB200 NVL72 rack is designed for well over 100 kW. Very few existing enterprise server rooms can deliver that density, and retrofitting one takes longer than most AI roadmaps allow.
The second is lead time. Current-generation GPUs reach India on vendor allocation schedules measured in months, so a provider that already holds inventory in-country compresses your timeline from quarters to weeks.
2. Where the data lives
For a bank, an insurer, a PSU or a government department in India, this is settled by law and by regulators. The Digital Personal Data Protection Act, 2023 governs how personal data is processed and moved. RBI has required payment system data to be stored in India since 2018. Sector regulators keep adding expectations around AI model governance and auditability. Our knowledge base article on DPDPA and the sovereign cloud covers the legal side in more depth.
The design consequence is simple. Training data, fine-tuned model weights and inference logs all need to sit in Indian data centres under controls you can show an auditor. That rules out quietly pushing a training set to an overseas region because it was cheaper that week. It also means the factory’s data layer must carry lineage from day one: which dataset version trained which model, who approved it, and when it was retired.
Sovereign capacity on a MeitY-empanelled cloud, or a community cloud built for BFSI or government workloads, gives you most of this by default. If you build on general-purpose capacity instead, budget for the compliance work as a separate line. It will not be a small one.
3. How much is training, how much is inference
Most enterprise AI factories are inference factories. Unless you are training a foundation model from scratch, which very few organizations in India should attempt, training is an occasional burst and inference is the daily grind. Fine-tuning a 7 billion or 13 billion parameter model on your own documents might take a few hours on a handful of GPUs each month. Serving that model to ten thousand employees runs all day, every day.
The hardware follows from this. Inference at scale is bound by memory bandwidth and latency, and it is usually served well by the L40S, or by the H200 where model size or memory bandwidth demands it, both well below the cost of a Blackwell training system. Large-scale training and very large models are where B200, B300 and GB200 NVL72 systems earn their keep. The NVLink fabric between GPUs is what lets a model that does not fit on one card train efficiently across dozens.
Size the two pools separately, then check the ratio against the demand sheet. If 80 percent of your projected GPU-hours are inference, buying a training-grade SuperPOD (NVIDIA’s reference design for a large multi-node GPU cluster) for the whole estate means paying for interconnect you will rarely use.
4. The platform layer between the GPUs and the users
Raw GPUs are not a factory. The platform is what makes the line repeatable, and it has a known shape:
- A scheduler and container platform, usually Kubernetes with GPU-aware scheduling, so capacity is shared between teams instead of hoarded.
- A data layer that keeps every dataset versioned, with a feature store for tabular models and a vector index for the documents that language models retrieve from.
- A model registry that records every trained version, its evaluation scores and its approval status.
- Pipelines that carry a model from training through evaluation into production, with a sign-off step and a tested way back to the previous version.
- Observability across the whole estate: GPU utilization, inference latency, cost per request and model drift (the slow loss of accuracy as live data moves away from the training data), feeding the same operations tooling your infrastructure team already watches. Our knowledge base covers how ESDS delivers and monitors GPU for enterprise AI and what AIOps solves in modern infrastructure.
You can assemble all of this from open-source parts. Many teams do and then spend their first year maintaining the assembly rather than shipping models. The alternative is a managed platform where the provider handles scheduling, registry management and monitoring, allowing your team to focus directly on building and running pipelines. For a first factory the managed route is usually faster to value and easier to audit. The do-it-yourself route makes sense once you have an engineering bench large enough to own it.
5. Governance and the operating model
Most teams defer this decision, and it is the one that decides whether the factory survives its first audit. Four things need to be agreed before the first model ships.
- Who owns each model in production: a named business owner who can decide to retrain, pause or retire it, rather than the data scientist who built it.
- What the approval gate checks at each level of model risk.
- Who runs the shared standards, pipeline and registry.
- How GPU cost is attributed to the teams that consume it.
Showback (reporting each team’s consumption) or chargeback (billing it) from the first month changes behaviour more than any policy memo. The next section sets out the operating model that answers the other three.
Who runs the factory: The Centre of Excellence model
A factory needs a permanent operator, and the first AI programme in most organizations does not have one. It has a project: a sponsor, a budget, a delivery team and an end date. That carries the first use case and fails at the second, because the standards, the pipeline and the lessons leave with the project team.
The structure that holds up is an AI Centre of Excellence (COE): a small standing team that owns the standards and the shared platform, while business units own the use cases and the outcomes. The split matters. A COE that builds every model becomes a bottleneck. A COE that owns nothing becomes a committee.
| Who | Owns | Does not own |
| AI Centre of Excellence | Governance policy, model risk tiers, the approval gate, templates for model cards and evaluation reports, the shared pipeline and registry, the reuse catalogue, the monthly scorecard, training for business teams | Individual use cases or their business results |
| Business unit | The use case, its demand numbers and value estimate, the data owner, the named owner of each model in production | The platform or the standards |
| Infrastructure and platform team | GPU capacity, scheduling, monitoring, cost showback | Model approval |
| Risk and compliance | Independent review of higher-tier models, the audit trail, regulator and auditor responses | Delivery timelines |
Tier the models, or the gate gets bypassed
Not every model deserves the same gate, and one heavy gate for everything is how governance ends up being worked around. Three tiers cover most estates.
| Tier | Typical models | What the gate requires |
| Tier 1: affects an individual’s money, access or rights | Credit decisions, claims adjudication, KYC rejection, benefit eligibility, hiring screens | Independent review by risk, bias and fairness testing on protected attributes, an explainability note, a human override path, quarterly re-evaluation |
| Tier 2: operational decisions with business impact | Demand forecasting, predictive maintenance, network optimization, fraud alerts routed to an analyst | COE review, evaluation on held-out data, monitoring thresholds with alerting, a tested rollback |
| Tier 3: internal productivity on non-sensitive data | Knowledge assistants, code assistants, document summarization | Self-certification against the checklist, usage logging, periodic sample review by the COE |
The evidence pack and the cadence
For each production model keep one evidence pack an auditor can read without a walkthrough: a model card, dataset lineage, the evaluation report, the approval record with names and dates, and the monitoring log. Organizations that already hold ISO/IEC 27001 will recognise the shape. ISO/IEC 42001, the management system standard for AI published in late 2023, formalizes it, and aligning the factory to it early is far cheaper than retrofitting for certification later. In India the pack should also show how personal data in the pipeline is handled under the DPDP Act, since that is the first question a sector regulator asks, and it must be mapped to MeitY’s AI governance guidelines.
Governance that happens only at approval time decays. The cadence that works has three parts. A monthly COE forum reviews the scorecard, the exceptions and the models due for re-evaluation. A quarterly review keeps the policy current. An annual internal audit checks the state against the policy, run by someone outside the COE.
ESDS runs its own AI estate on the same pattern. An enterprise AI governance policy is owned by the Chief AI Officer. An AI Governance Office runs the gates and keeps the evidence. A cross-functional COE forum owns process standards across the organisation. The structure typically takes a quarter to put in place, which is why it belongs in the readiness and foundation phases rather than after the first incident.
How to build an AI factory in phases
A first production use case in three to four months is realistic on managed capacity. A factory that ships new models without a bespoke project each time takes most of a year. Owning hardware adds procurement and facility time on top of this.
| Phase | Typical duration | What gets built | Exit gate |
| 0. Readiness | 4 to 6 weeks | Readiness checklist, demand sheet, data audit of the first two use cases, compute sizing, draft governance policy | Demand sheet and budget signed off; a named data owner per use case |
| 1. Foundation | 8 to 12 weeks | First GPU pool (managed or dedicated), GPU-aware scheduling, model registry, one end-to-end pipeline for use case one | Use case one in production with monitoring and rollback |
| 2. Repeatability | 3 to 4 months | Use cases two and three on the same pipeline, evaluation gates automated, cost show back live | A new model reaches production without a bespoke project |
| 3. Scale | Ongoing | Multi-team access, self-service environments, a training pool if the demand sheet justifies it, retraining triggered by drift | Utilisation above target; audit evidence produced on request |
Five numbers tell you whether you have a factory or a lab:
- GPU utilization across the estate. A mature factory runs well above half; a first-year one rarely does without active scheduling.
- Lead time from an approved use case to production. Measured in weeks.
- Cost per thousand inferences, or per million tokens for language models, trending down quarter to quarter.
- Share of production models with a named owner, data lineage and an approval record. This should be 100 percent, and auditors will check.
- Time from drift detected to model retrained and redeployed. Measured in days.
If you can report these five on one pages every month, you have a factory.
Mistakes that turn a factory back into a pilot
We see the same handful of errors across sectors. None of them are about hardware.
1. Buying for the largest model anyone can imagine.
The capacity sits idle for the first year while the team ships a 7 billion parameter assistant and a tabular fraud model. Buy for the demand sheet and keep the option to grow.
2. Letting every team build its own pipeline.
Three MLOps stacks, three registries, nothing reusable, and the second team repeats every mistake the first one made. One shared pipeline is the whole point.
3. Treating data residency as a procurement checkbox
A tick on a tender form does not stop an engineer from spinning up a training job in an overseas region at 2 am. Residency must be enforced by the platform; a document cannot do it.
4. Skipping cost attribution
When GPU-hours are free for the team using them, the team using them will use all of them. Attribution from month one is the cheapest control there is.
5. Measuring the factory by models trained
The only number that matters to the business is decisions made, and the cost and quality of each. A factory that trains fifty models and serves two is a research lab with a large power bill.
Frequently asked questions about building an AI factory.
How long does it take to build an AI factory?
On managed GPU capacity, a first use case can be in production in three to four months, with a repeatable multi-team factory taking nine to twelve months. Buying and installing your own GPU hardware adds procurement, power and cooling work on top, which in India can mean another two quarters. The long pole is rarely the GPUs; it is data access approvals and the first governance gate, which is why the readiness phase starts with both.
How much does an AI factory cost in India?
The cost is driven far more by how the GPUs are used than by which GPUs are bought. A managed inference pool serving two or three use cases costs a small fraction of a captive training cluster, and idle capacity is the largest avoidable expense in most first-year budgets. For context, the IndiaAI Mission makes subsidized compute available to start-ups, researchers and academia at about Rs 65 per GPU-hour (PIB); enterprises pay commercial rates, so utilization discipline matters more than the sticker price.
Do I need to own GPUs to build an AI factory?
No. Start on managed capacity, measure real utilization and latency for the first two use cases, and move a workload onto a dedicated pod only when the demand sheet or a regulator requires it. Most Indian enterprises end up with a small, dedicated pod and rented capacity for everything else.
What is the difference between an AI factory and GPU as a Service?
GPU as a Service is the compute supply: accelerators, interconnect and storage delivered on demand. An AI factory is the whole production system built on top of that supply, including the data layer, pipelines, model registry, monitoring and governance. GPU as a Service is normally one component of an AI factory, not a substitute for it.
Can a mid-sized company build an AI factory?
Yes, at a smaller scale. The same discipline applies: a demand sheet, one shared pipeline, a model registry and an approval gate. Managed platforms make this realistic for a team of five, because the scheduling, monitoring and registry are run for you.
How does data sovereignty affect AI factory design in India?
It decides where the factory can physically run. Personal data under the DPDP Act, payment data under RBI’s 2018 localisation rule and anything a sector regulator may audit all have to stay on Indian infrastructure, so the GPU pool, the data layer and the logs are placed in India from the start. Sovereign and community clouds built for BFSI and government exist for exactly this reason.
Do we need an AI Centre of Excellence to run an AI factory?
For more than one or two use cases, yes, although it can start as a team of three. The COE owns the standards, the shared pipeline, the model registry and the monthly scorecard, while business units keep ownership of their use cases and outcomes. Without one, every team rebuilds the pipeline and the second audit finds three different approval processes.
Which standards apply to AI factory governance in India?
The DPDP Act, 2023 for personal data; sector guidance from RBI, IRDAI or SEBI for regulated firms; MeitY’s India AI governance guidelines; CERT-In’s security directions; and ISO/IEC 42001 as the auditable management system standard for AI. Most organisations align the factory’s evidence pack to ISO/IEC 42001 so that an existing ISO/IEC 27001 audit programme can extend to cover AI.
Where to start
The hardware is the easiest part of an AI factory to buy and the hardest part to use well. The organisations getting value in year one start from a demand sheet and rent before they own. They keep data inside India by design and run every model through one pipeline. They put a governance gate in front of production before the first model asks for it, and they give a Centre of Excellence the job of holding the standard.
ESDS runs an NVIDIA-validated GPU SuperPOD in India, with GPU options spanning L40S and H200 for inference, B200, B300 and GB200 NVL72 for training, and AMD MI300X. Delivery follows the demand sheet: a managed GPU cloud, dedicated managed infrastructure, or a captive cluster we help design, supply and deploy for organisations that must own their own. Regulated workloads can sit on ESDS community clouds built for BFSI and government, with the data layer and controls in Indian data centres from the start.
If you have a demand sheet, or want help writing one, talk to an ESDS AI architect. And if you are still at the readiness stage, our knowledge base checklist on whether your organisation is ready for an AI factory is the place to start.
- How to Build an AI Factory in India: A Practical Blueprint - October 15, 2026
