AI Brief: hbf-data-manager

NestJS service that persists per-interaction metadata produced by hbf-bot during workflow execution. Writes come in via Kafka; reads are served over a JWT-guarded REST API.

What This Repo Does

hbf-data-manager consumes interaction-metadata events from Kafka (published by hbf-bot) and stores them in MySQL as InteractionMetadata rows. It exposes a read API for org users (guarded by hbf-core token validation) to list, filter, and inspect those records, and a write-delete API (guarded by a local DM-issued JWT) used by hbf-data-retention to clean up sessions.

Tech Stack

• Language: TypeScript
• Framework: NestJS 11 (Express platform)
• Key dependencies: TypeORM 0.3 (MySQL), kafkajs 2.2, @helvia/hbf-core-api 30, @nestjs/jwt, nestjs-pino

Entry Points

• Main: src/main.ts
• App module: src/app.module.ts
• ORM config: ormconfig.ts (migration CLI only)

Key Directories

Directory	Purpose
src/interactions/	InteractionMetadata entity, read controller, delete controller, service, DTOs
src/queue/	Kafka consumer: subscribes to topics and calls InteractionsService.saveMetadata
src/auth/	Two guards: HBFGuard (delegates to hbf-core), DMJwtGuard (local HS256 JWT)
src/clients/	CoreClientService: HTTP client wrapping hbf-core /users/me
src/migrations/	TypeORM migration files for interaction_metadata table
src/utils/	Shared DTOs (User, UserRole) and sleep helper
src/health/	Health check controller on GET / and GET /health

API Endpoints

Method	Path	Guard	Description
GET	/orgs/:orgId/interactions	HBFGuard	List all interaction metadata (desc by ts)
GET	/orgs/:orgId/events/:hbfEventId/interactions	HBFGuard	Get records for a specific HBF event (sortable)
GET	/orgs/:orgId/events/:hbfEventId/session-variables	HBFGuard	Latest VARIABLE-type records as of event timestamp
DELETE	/chat-sessions/:sessionId/interactions	DMJwtGuard	Async delete all records for a session (202)
POST	/chat-sessions/interactions/bulk-delete	DMJwtGuard	Async bulk delete by session IDs (202)
GET	/health	none	Service health check

Data Model

Table: interaction_metadata

Column	Type	Notes
id	BIGINT PK	Auto-increment
hbf_event_id	CHAR(36)	HBF event UUID
session_id	CHAR(36)	ChatSession UUID
ts	TIMESTAMP	Interaction timestamp
type	TEXT	E.g., "VARIABLE", "NODE", etc.
friendly_name	TEXT nullable	Human-readable node label
payload	JSON	Event-specific data
has_error	BOOLEAN nullable	Whether the interaction errored
duration_ms	INT nullable	Duration in milliseconds
created_at	TIMESTAMP	Row creation time
updated_at	TIMESTAMP	Row last updated time

Indexes: idx_session_ts (session_id, ts), idx_hbf_event_ts (hbf_event_id, ts)

External Dependencies

• MySQL 8: Primary data store (hbf_data_manager database)
• Kafka: Consumes interaction-metadata topic (configurable via KAFKA_TOPICS). Supports local PLAINTEXT and Confluent Cloud SASL_SSL.
• hbf-core: Token validation via GET /users/me (env: CORE_URL)

Auth

Two guard patterns:

1. HBFGuard (read endpoints): Calls hbf-core GET /users/me with the incoming Bearer token. Allows moderators unconditionally; allows org-role holders for the requested orgId.
2. DMJwtGuard (delete endpoints): Verifies a locally-issued HS256 JWT signed with DM_AUTH_JWT_SECRET. Used by internal services (e.g., hbf-data-retention) to delete session data.

Key Environment Variables

Variable	Default	Purpose
MYSQL_HOST	localhost	MySQL host
MYSQL_PORT	3306	MySQL port
MYSQL_USER	root	MySQL username
MYSQL_PASSWORD	root	MySQL password
MYSQL_DB	hbf_data_manager	MySQL database name
CORE_URL	http://localhost:8080	hbf-core base URL
KAFKA_BROKERS	localhost:9092	Comma-separated Kafka brokers
KAFKA_TOPICS	interaction-metadata	Comma-separated topics to consume
KAFKA_GROUP_ID	hbf-data-manager-consumer	Consumer group ID
KAFKA_SECURITY_PROTOCOL	PLAINTEXT	PLAINTEXT or SASL_SSL
KAFKA_SASL_USERNAME	—	Confluent Cloud username
KAFKA_SASL_PASSWORD	—	Confluent Cloud password
DM_AUTH_JWT_SECRET	—	Secret for DMJwtGuard (delete API)
TYPEORM_AUTORUN_MIGRATIONS	false	Auto-run migrations on startup
DB_SYNCHRONIZE	false	TypeORM schema sync (keep false in prod)
PORT	3000	HTTP listen port
PINO_LOGGER_USE	false	Enable Pino structured logging
PINO_LOG_LEVEL	info	Pino log level

Running Locally

# Start MySQL only
docker compose up -d db

# Run with hot reload
MYSQL_HOST=localhost MYSQL_PORT=3306 MYSQL_USER=hbf MYSQL_PASSWORD=hbf MYSQL_DB=hbf_data_manager npm run start:dev

API: http://localhost:3000 Swagger: http://localhost:3000/api (non-production only)

Migrations

npm run migration:run    # apply pending migrations
npm run migration:revert # revert last migration

Tests

npm test          # unit tests
npm run test:e2e  # e2e tests
npm run test:cov  # coverage