Intranet Sync Design

This document explains the architecture and design rationale behind the intranet data synchronization system.

Problem

The Wippidu Kita App needs to receive organizational data (locations, groups, children, employees, relationships) from an external intranet management system. This data must be imported safely without disrupting the running application.

Two-phase architecture

The sync system uses a deliberate two-phase design:

┌─────────────────┐      ┌──────────────────┐      ┌─────────────────┐
│  Wippidu        │      │    Wippidu App   │      │   Wippidu App   │
│  Intranet       │─────►│  Staging Tables  │─────►│   App Tables    │
│  (External)     │ API  │  (sync_*)        │Admin │  (locations,    │
└─────────────────┘      └──────────────────┘ UI   │   groups, etc.) │
                                                    └─────────────────┘

Phase 1 — Data Reception (API): The external intranet system pushes data to staging tables via authenticated REST API. This is fully automated.

Phase 2 — Data Processing (Admin): An admin reviews pending data and triggers processing, which transforms staging records into application tables. This is a deliberate manual step.

Why two phases?

1. Safety: Data from external systems is untrusted. Staging tables act as a buffer, allowing admins to review before affecting the live application.
2. Atomicity: Processing runs in dependency order (locations → groups → children → assignments). If one step fails, earlier steps are already committed and can be retried.
3. Audit trail: All processing operations are logged with timestamps, counts, and the admin who triggered them.
4. Decoupling: The API can accept data at any time without worrying about processing dependencies. Processing happens when the admin is ready.

Data flow detail

External Intranet System
         │
         │ HTTP POST + Bearer Token
         ▼
┌─────────────────────────────────────┐
│         API Authentication          │
│   (middleware/api_token.go)         │
│   • Validate Bearer token           │
│   • Check IP allowlist              │
│   • Update last_used_at             │
└─────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│         API Handlers                │
│   (controller/intranet_api.go)      │
│   • Parse JSON payload              │
│   • Upsert to staging tables        │
│   • Return sync result              │
└─────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│         Staging Tables              │
│   sync_persons, sync_groups, etc.   │
└─────────────────────────────────────┘
         │
         │ Admin triggers processing
         ▼
┌─────────────────────────────────────┐
│         Processing Service          │
│   (service/sync_processor.go)       │
│   • Process in dependency order     │
│   • Match by ExternalID             │
│   • Create/update app records       │
│   • Log results                     │
└─────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│         App Tables                  │
│   locations, groups, children       │
└─────────────────────────────────────┘

Processing dependency order

Processing must occur in this order because of foreign key relationships:

1. Locations      (no dependencies)
       │
       ▼
2. Groups         (depends on Location via EinrichtungsID)
       │
       ▼
3. Children       (no dependencies, but logically after groups)
       │
       ▼
4. Child Groups   (depends on Child + Group)

ExternalID mapping

App tables use ExternalID fields to match staging records to existing app records. This enables upsert behavior:

• If a matching ExternalID exists → update the record
• If no match → create a new record

This pattern is implemented in SyncProcessor for each entity type.

What gets processed vs. what doesn't

Directly processed (staging → app tables):

Staging	App Table	Notes
sync_locations	locations	Direct mapping
sync_groups	groups	Requires location dependency
sync_persons (rolle=1)	children	Children only
sync_belegungen (status=3)	children.group_id	Updates group assignment

Used by registration flow (not direct processing):

Staging	Usage
sync_employees	Employee registration lookup
sync_persons (rolle=2,3,4)	Parent registration lookup
sync_parent_children	Link parent to children during registration
sync_location_leads	Assign LocationLead role during registration
sync_group_leads	Assign GroupLead role during registration

Components

Component	Location	Purpose
API Token Model	internal/model/sync.go	Token authentication storage
Staging Models	internal/model/sync.go	Staging table definitions
Auth Middleware	internal/middleware/api_token.go	Bearer token + IP validation
API Controller	internal/controller/intranet_api.go	Receive and store data
Processing Service	internal/service/sync_processor.go	Transform staging → app tables
Admin Controller	internal/controller/admin_sync.go	Trigger processing via UI
Routes	internal/route/intranet.go	API endpoint definitions

Daily employee group sync

In addition to the batch sync mechanism above, the app supports real-time daily group assignments for employees. This handles scenarios where employees substitute in different groups day-by-day.

How it works

┌─────────────────┐      ┌──────────────────┐      ┌─────────────────┐
│  Employee       │      │   Wippidu App    │      │   Intranet      │
│  Logs In        │─────►│   (on login)     │─────►│   Groups API    │
└─────────────────┘      └──────────────────┘      └─────────────────┘
                                │                          │
                                │◄─── JSON response ───────┤
                                │     with group IDs       │
                                ▼
                         ┌──────────────────┐
                         │  employee_daily  │
                         │  _groups table   │
                         └──────────────────┘

1. Login trigger: When an intranet-linked employee logs in, the app checks if they've been refreshed today
2. API call: If not, the app calls the intranet API with the employee's Mitarbeiter_ID
3. Response processing: The API returns group assignments with Stammteam (core team) flags
4. Priority logic: Substitute groups (Stammteam=false) take precedence over core team groups
5. Storage: Assignments are stored in employee_daily_groups with today's date

Viewing vs. creating

The daily sync creates a deliberate separation between access levels:

Permission	Source Function	Groups Included
View children	GetEmployeeGroups()	All daily groups (substitute or core)
Create content	GetEmployeeCoreTeamGroups()	Only core team groups

This means: - A substitute can see children in their substitute group - A substitute cannot create news/letters/events for that group - Creation rights stay with their home (Stammteam) group

Why this design?

1. Real-time accuracy: Staff schedules change daily; batch sync isn't fast enough
2. Minimal permissions: Substitutes shouldn't modify content for groups they don't normally manage
3. Audit clarity: Content is always created by permanent staff, not temporary substitutes

Related

• Intranet Sync API Reference — Endpoint documentation
• How to Sync Intranet Data — Practical guide