Evaluating an AI HRMS in 2026: a 12-question checklist

Most AI HRMS evaluations turn into vendor demos with marketing slides instead of real answers. Here are 12 questions that separate AI HR products that work for Indian companies from the ones that don't — including the questions you should ask us.

"AI HRMS" is the new "cloud-native HRMS" was in 2014. Every vendor claims to be one. The category descriptions in their proposals are interchangeable. The screenshots all show some variation of a chatbot icon hovering in the corner of the dashboard.

If you're evaluating AI HR products for an Indian company in 2026, the hard part isn't shortlisting candidates. Plenty of vendors will get on a call. The hard part is asking questions specific enough that the AI generic-marketing-talk falls apart, and what's actually behind the product becomes visible.

This piece is 12 such questions. They're not "what features does the AI have" — features lists are easy to fake and easier to ignore. They're questions about where the AI lives, who controls it, what happens when it fails, and whether it understands Indian payroll context. A vendor with good answers to these 12 may or may not be the right fit for your company. A vendor with bad answers definitely isn't.

Run these against every vendor on your shortlist. Including us.

Part 1 · Where the AI actually lives

Most "AI HR" features in 2026 are wrappers around third-party large language models — OpenAI, Anthropic, Google, Meta's open-weight Llama family. That's not a problem in itself. But it changes the answers to compliance, residency, and pricing questions in ways most vendors don't volunteer.

QUESTION 01

Where does inference physically run?

Compliance · Data residency

If your AI HRMS sends employee data to a US-hosted LLM endpoint to process payroll anomalies or summarise leave requests, that's a cross-border transfer under DPDP Section 16. Even if the data comes back in milliseconds, it left India to be processed.

Good answer

"Inference runs on AWS Mumbai (ap-south-1) using a region-locked endpoint. Data does not leave India. We can show you the AWS region in the network logs."

Bad answer

"We use OpenAI/Anthropic for processing." Both of those endpoints are US-hosted by default. Following up with "but it's all encrypted" doesn't fix the residency issue.

Red flag Vendor can't answer the question without escalating to engineering. If the founder or sales lead doesn't know where inference runs, no one has thought about this carefully.
QUESTION 02

Who owns the prompts and the outputs?

IP · Data ownership

When the AI processes a leave request and writes a summary, who owns that summary? When you ask the AI HR assistant a question, who owns your question text? The vendor's contract should make this explicit.

Good answer

"You own your prompts and your outputs. The contract assigns IP in user-generated content to you, and we keep no extraction rights."

Bad answer

"All AI-generated content is owned by [vendor name]." That's an outright IP grab. Do not sign that contract.

Red flag The Master Services Agreement doesn't mention prompts or outputs at all. That ambiguity will resolve in the vendor's favour during a dispute.
QUESTION 03

Is our data used to train your models?

Training data · Privacy

The question every AI vendor sidesteps. There are three honest answers: "yes, with opt-out", "no, never", or "no, but our LLM provider might". Each has different implications.

Good answer

"No. We use the LLM provider's enterprise tier with data-not-used-for-training contractually guaranteed. We can show you the enterprise agreement clause."

Bad answer

"We anonymise it before training." Anonymisation of HR data is famously weak. Salary plus designation plus city plus tenure can re-identify most employees."

Red flag "We don't know, you'd have to ask OpenAI." If they're using a consumer-tier API, they don't have the answer. Walk away.
QUESTION 04

What's the model architecture under the hood?

Technical

You don't need a PhD answer. You need enough specificity to verify the claim. "AI" alone covers everything from a regex-rules engine to GPT-5; the differences matter.

Good answer

"Anomaly detection uses XGBoost on historical payroll data. The HR assistant uses Claude Sonnet via Anthropic's enterprise API, with retrieval over your tenant data."

Bad answer

"We use proprietary AI, deep learning, and machine learning algorithms." That's a non-answer in three nouns.

Red flag Refusal to name the underlying model on the grounds of "trade secret." Architecture isn't a trade secret. The training data and prompts might be.
QUESTION 05

What's the failure mode when the AI is wrong?

Reliability

Every AI system gets things wrong. The question is what happens when it does. Does the AI silently produce a wrong payroll calculation, or does it flag uncertainty and route to a human?

Good answer

"For payroll anomalies, the AI flags candidates and a human reviewer approves before processing. For HR Q&A, the AI cites its sources from your policy documents and says 'I don't know' when it lacks context."

Bad answer

"Our AI is 99% accurate." That's the wrong metric. The question is what the 1% looks like and whether the system fails loud or silent.

Red flag No clear human-in-the-loop boundary. If the AI can independently approve payroll, terminate employees, or make hiring decisions, the vendor has confused "automation" with "agency."

Part 2 · How the AI behaves operationally

Once you're satisfied the AI is hosted in a defensible way, the next layer of questions is about how it behaves day-to-day. Bias, audit, contract exit.

QUESTION 06

How do you monitor for bias?

Fairness · India-specific

In an Indian context, bias is more than gender. Caste signals embedded in surnames, regional language patterns, age, employment gaps that correlate with maternity — all of these can leak into AI recruitment and performance systems without anyone intending it.

Good answer

"We do periodic fairness audits on outcomes — interview pass rates, performance ratings, attrition flags — across protected characteristics. We can share the methodology."

Bad answer

"Our model is unbiased because we removed demographic features from training." Removing the obvious features doesn't help; proxies remain. Pin code alone correlates strongly with caste and income in many Indian cities.

Red flag "We let the customer configure their own bias controls." That's the vendor outsourcing responsibility for an unmanageable problem. Expect them to have a baseline themselves.
Worth knowing

India doesn't yet have an "AI Act" equivalent to the EU's, but the Digital India Act (in draft as of 2026) is expected to include algorithmic accountability obligations. If the vendor's posture is "we'll deal with it when the law arrives," they'll be late. Buyers should treat the EU AI Act's "high-risk" categories — recruitment, performance management, termination decisions — as a reasonable preview of where Indian law is heading.

QUESTION 07

What does the audit trail look like?

Auditability · DPDP

If a regulator or an internal auditor asks "why did the AI flag this employee for attrition risk?", the vendor needs to give an answer that satisfies the question. Black-box "the model said so" is not an answer.

Good answer

"Every AI decision is logged with the input features, the model output, the confidence score, and the action taken. We can export the audit log on demand."

Bad answer

"The audit log captures user actions." That's UI-level logging. What you need is decision-level logging — what the AI did and why.

Red flag No way to retrieve the reasoning for a specific AI decision after the fact. If you can't reconstruct why the system flagged Priya as a flight risk last quarter, you can't defend it.
QUESTION 08

What happens at contract end?

Exit · Data portability

If you cancel in year 2, what happens to your data? What happens to any models trained or fine-tuned on it? This is uncomfortable for vendors to discuss in year 0, which is exactly why you should bring it up.

Good answer

"On contract end, we give you a full data export in standard formats (CSV, JSON), delete your tenant data within 30 days, and certify deletion in writing. Any tenant-specific fine-tuning is also deleted."

Bad answer

"Standard SaaS exit terms apply." That's not specific enough. Ask for the specific timeline and the specific export format.

Red flag Reluctance to commit a deletion timeline in writing. The Indian SaaS market has many vendors who hold data hostage during exit negotiations.
QUESTION 09

How is AI priced?

Pricing · Predictability

The AI feature you saw in the demo might be free, an add-on per-seat, or metered per-call. Each one creates a different cost profile as you scale.

Good answer

"Per-seat add-on at a transparent monthly rate, with a fair use ceiling spelled out. No usage-based pricing for core features."

Bad answer

"It's metered — Rs.X per AI call." Pay-per-call pricing creates anxiety. People stop using the AI to avoid the bill, which defeats the purpose. Avoid pure usage-based pricing for AI HR.

Red flag "Our AI is included free for now." Free-for-now means the price comes later. Ask what the future state pricing will be.

Part 3 · Does the AI understand India?

The biggest hidden weakness of most "AI HRMS" products is that they were built for a global market and bolted on Indian payroll features. The AI gets confused by things every Indian HR person finds obvious — sandwich leave, half-yearly PT for Tamil Nadu and Kerala, the ESI ₹21,000 ceiling, gratuity at 4 years 8 months versus 5 years.

These three questions test whether the vendor has thought about India as a market, not as an add-on.

QUESTION 10

What's your DPDP compliance posture?

India · DPDP

Under the Digital Personal Data Protection Act 2023, you (the employer) are the Data Fiduciary for employee data and the AI HRMS vendor is typically the Data Processor. The DPA between you should reflect this — and the vendor should have one ready.

Good answer

"We're a Data Processor for your employee data. We have a standard DPA available for review. We've named a Grievance Officer and published their contact under DPDP Section 8(10)."

Bad answer

"DPDP doesn't apply to us because we're a tech vendor." DPDP applies to every Data Fiduciary and Data Processor handling personal data in India. "We're tech" isn't an exemption.

Red flag No DPA on file. No named Grievance Officer. The vendor hasn't read the Act.
QUESTION 11

Does the AI understand Indian statutory context?

India · Statutory accuracy

Ask the AI HR assistant a question that requires Indian statutory knowledge. "What's the PF ceiling for FY 2026-27?" Or "How is sandwich leave handled?" Or "Is gratuity payable at 4 years 8 months?" A US-trained LLM with a thin India layer will give wrong, confident answers.

Good answer

"Try it during the demo. We'll give you a list of 20 Indian-specific questions; you'll see how it answers and whether it cites the right Acts and sections."

Bad answer

Live demo answer that says "The PF ceiling is $1,800" or "The Family and Medical Leave Act applies." Both are real failures we've seen.

Red flag The AI gives confident wrong answers to Indian statutory questions. Confident wrong is worse than admitting "I don't know." Wrong answers create compliance liability you didn't budget for.
QUESTION 12

Does the AI work in Hindi and regional languages?

India · Language

If you have field staff, factory workers, retail employees, or call centre teams, English-only AI is a non-starter. The product needs to work in Hindi at minimum, ideally in Tamil, Telugu, Marathi, Bengali, Kannada, and Malayalam too.

Good answer

"The ESS app and HR assistant work in 8 Indian languages. Auto-translation for leave request reasons. Voice input in Hindi."

Bad answer

"English only, with localisation on the roadmap." For a product launching in 2026, English-only is a 2018 product. The roadmap excuse means it's not coming soon.

Red flag "We use Google Translate for non-English." Google Translate is fine for menus. It's not fine for legally-binding policy text or for nuanced employee communication.

How to actually run this evaluation

Twelve questions sounds like a lot. In practice, you can ask all of them in a 60-minute call if the vendor has thought about them, and you'll never get past question 4 if they haven't. Here's a process that fits inside your existing demo cycle:

Before the demo: send the 12 questions in writing. Let the vendor prepare. You're not trying to catch them off guard; you're trying to see whether the answers exist at all. Two days' preparation is fair.

During the demo: ask follow-up questions on the three answers that surprised you most — good or bad. Use the live demo to test Question 11 (statutory context) in real time.

After the demo: ask for the answers in writing as part of the proposal. "Send me your written answers to questions 1, 3, 7, 8, and 10" is a reasonable request. If the vendor pushes back on putting answers in writing, that itself is the answer.

Across vendors: compare answers side-by-side. The differences will be larger than you expect. Specifically watch for vendors who use the same phrasing — that often means they all rebadged the same underlying model and are competing on UX rather than substance.

Try these 12 questions on hrPLANR

Book a 30-minute demo and ask us all 12. We'll give you written answers afterwards. If our answers are bad, you'll know in an hour instead of 6 months.

Book a demo →

One closing observation

The single biggest predictor of whether an "AI HRMS" works for an Indian company isn't the model architecture, the pricing, or even the statutory accuracy. It's whether the founding team has run a real Indian payroll cycle themselves. Not "advised on" or "consulted for" — actually run it. Watched the gross-to-net reconciliation. Filed a 24Q. Fought with the EPFO portal at 11:45 PM on the 14th.

Those founders build products that handle the actual edge cases. Founders who've only seen Indian HR from Bay Area conference rooms build products that handle the slides about Indian HR.

Ask, during the demo, who on the team has done payroll in India. The answer tells you which kind of product you're being sold.


This article reflects the state of Indian AI HRMS market as of May 2026. Model architectures, regulatory landscape, and vendor offerings are evolving — we re-review this article quarterly and publish updated questions when warranted. Not legal advice.

Last updated: 26 May 2026.