INI-001
·Aprovada p/ PoCHR assistant for benefits questions
Marina Costa · HRRAG / KnowledgeCriticality MediumAutonomy: Assistive
84Overall
Automatic evaluation
Business Value82
AI Quality78
Security88
Privacy90
Opinion
Well-scoped use case, internal-only and low risk. RAG is the right choice — no need for fine-tuning. Recommended to move to PoC with 100 users.
Aieval qualifies and recommends — the final decision is always human.
Gaps
1- Knowledge base update plan not defined
Risks
2- Stale answers if policies change without reindexing
- Possible hallucination on out-of-scope questions
Recommendations
3- Define a weekly knowledge base reindexing cadence
- Add an 'I don't know' guardrail when confidence < 0.6
- Include thumbs up/down feedback for continuous tuning
Discovery
Answers collected during intake
- Problem
- HR team receives ~400 repeated questions/month about benefits, vacation and meal vouchers.
- Objective
- Reduce level-1 ticket volume by 60% with instant answers grounded in official policies.
- Impacted users
- ~8,500 internal employees
- Vendor / model
- OpenAI (GPT-4o) via Azure · LLM + RAG over policy base
- Data used
- Internal HR policies, Benefits FAQ
- Integrations
- SharePoint, Microsoft Teams
- KPIs
- Ticket deflection (%) · Internal CSAT · Average response time
- Estimated cost
- $12,000
- Risk flags
- Uses PII: NoCustomer-facing: NoProduction access: NoExternal sharing: No