Skip to content
CASE STUDY

Eleven services, three engineers

Working across HCMatrix: one question for choosing gRPC or RabbitMQ, a cache in the right place, and clock-ins that arrived twice.

HCMatrix is an HR platform used by more than 20 organizations, covering payroll, recruitment, attendance, performance and more. It runs as eleven Node.js services, and at SnapNet there were three of us on the backend. I worked across all of them.

With a team that size, you can’t afford a system that surprises you. These are the habits and fixes that mattered most.

One question: does the caller need the answer?

The platform already used gRPC and RabbitMQ, and the choice between them came down to one question.

If the caller can’t continue without the result, it’s a synchronous gRPC call. Every service asks one shared directory for company and employee data rather than keeping its own copy, and the approval workflow calls back into whichever service owns the record once a request is approved.

If nobody is waiting on the result, it goes through RabbitMQ. Notifications and downstream processing work this way, so a slow consumer doesn’t hold up the request that started the work.

Cache what’s read constantly and changes rarely

When response times suffered at peak, the data every service kept asking for was the obvious place to start. In an HR platform that’s mostly the directory: employees, companies, roles and permissions.

I put Redis in front of that, and not in front of everything, because a cache over frequently written data buys speed at the cost of correctness. I also optimized the slowest queries and added indexes based on real query patterns.

Throughput went up by about 30%, measured by comparing request-rate and latency panels in production monitoring before and after.

Anything that retries must be safe to receive twice

Biometric devices push clock-in and clock-out events to the attendance API, and they sometimes re-sent them. Employees ended up with duplicate attendance records.

SAME CLOCK-IN, SENT AGAIN
3events received
1attendance records
Employee 1042 / 08:59

I could have removed the duplicates in the reports. Instead I fixed it where events enter the system, so a re-sent event is recognised and ignored, and everything reading attendance sees one record. Any integration that retries has to be safe to receive twice.

The service I’m proudest of

HCMatrix needed a recruitment module, and its flow hadn’t been designed when the work started. I scoped it with the customer relations team and the other developers first, then built most of it: candidate and CV records, interview scheduling, shared evaluations, and candidate notifications over the existing RabbitMQ backbone. Later I added AI-generated interview questions on top.

I built it inside the existing architecture rather than as a special case, which is how three engineers keep eleven services understandable.

All case studies and posts