Skip to main content

Architecture: hbf-data-manager

Component Diagram​

Module Structure​

ModuleResponsibilities
AppModuleRoot: wires ConfigModule, TypeOrmModule (MySQL), LoggerModule, HealthModule, InteractionsModule, QueueModule
HealthModuleHealth check on GET / and GET /health
InteractionsModuleInteractionsController, InteractionsDeleteController, InteractionsService, InteractionMetadata entity
QueueModuleKafkaConsumerService — subscribes to Kafka on startup, calls InteractionsService
AuthModuleAuthService (JWT verify), JwtModule (DM_AUTH_JWT_SECRET)
ClientsModule (in auth)CoreClientService (HTTP to hbf-core), HttpClientService

Data Flow​

Write path (Kafka)​

hbf-bot publishes event to Kafka "interaction-metadata" topic
  -> KafkajsConsumer receives message
  -> KafkaConsumerService.onKafkaMessage parses JSON as InteractionMetadataDto
  -> InteractionsService.saveMetadata persists row to MySQL

Read path (REST)​

Client sends GET /orgs/:orgId/interactions (Bearer token)
  -> HBFGuard calls CoreClientService.getUserMe -> hbf-core /users/me
  -> Validates org-level role (or isModerator bypass)
  -> InteractionsService.findAll / findByHbfEventId / getLatestVariableRecordsAtEvent
  -> Returns DTOs

Delete path (internal)​

hbf-data-retention sends DELETE /chat-sessions/:sessionId/interactions (DM JWT)
  -> DMJwtGuard.canActivate verifies HS256 JWT (DM_AUTH_JWT_SECRET)
  -> InteractionsService.deleteBySessionId / deleteBySessionIds
  -> Returns 202 Accepted (async fire-and-forget)

Key Design Notes​

  • Dual auth pattern: read endpoints use HBFGuard (remote validation via hbf-core); delete endpoints use DMJwtGuard (local JWT, for internal services only).
  • Async deletes: delete controllers return 202 immediately and perform DB deletion in background via Promise.resolve().then(...).
  • Kafka SASL_SSL: buildKafkaConnectionConfig() detects KAFKA_SECURITY_PROTOCOL=SASL_SSL and configures kafkajs ssl: true + sasl: plain for Confluent Cloud.
  • Variable snapshot: getLatestVariableRecordsAtEvent reconstructs per-key variable state up to a given event timestamp by scanning all VARIABLE-type records in the session and keeping the latest per key.
  • Body limit: 5MB JSON body parser limit set in main.ts to handle large conversation payloads.