netvuln-tool ↔ Bullium Site API Contract

This document defines the integration contract between the netvuln-tool scanner (this repository) and the Bullium Consulting website collect portal (~/bullium-site/). Changes to either side must be coordinated.

Upload Endpoint

Property	Value
URL	POST /api/upload
Auth	x-api-key header
Env vars	CONSULTATION_API_URL, NV_API_KEY (or legacy CONSULTATION_API_KEY)
Config file	templates/report_config.conf.example
Max size	25 MB
Content-Type	multipart/form-data

Multipart Fields

Field	Type	Required	Source
session_json	application/json	Yes	session_results.json
pipeline_log	text/plain	No	pipeline.log
report_html	text/html	No	report_*.html
exec_summary_html	text/html	No	exec_summary_*.html

Success Response (HTTP 200)

{
    "success": true,
    "session_id": "20260301_120000",
    "share_token": "<64-char-hex>",
    "share_url": "https://bullium.com/share/?token=<64-char-hex>",
    "message": "Report submitted successfully"
}

Error Responses

Code	Meaning	Example
400	Bad request	Missing session_json, invalid JSON
403	Auth failure	Invalid or missing API key
413	Too large	Upload exceeds 25 MB
500	Server error	Storage failure

Refresh Endpoint

Property	Value
URL	PUT /api/refresh?session={id}&file={name}
Auth	x-api-key header
Body	Raw file content (text/html)
Allowed files	report.html, exec_summary.html

Replaces a single display file for an existing session without modifying metadata or other files. Used by refresh_reports.sh to bulk re-upload reports after template changes.

Success Response (HTTP 200)

{
    "success": true,
    "session_id": "20260301_120000",
    "file": "report.html",
    "message": "File refreshed successfully"
}

Error Responses

Code	Meaning	Example
400	Bad request	Missing session or file parameter, file not in whitelist
401	Auth failure	Invalid or missing API key
404	Not found	Session does not exist
405	Wrong method	Must use PUT

API Authentication

Legacy API Key (deprecated)

Property	Value
Header	x-api-key
Env var	NV_UPLOAD_API_KEY (server), CONSULTATION_API_KEY (CLI)
Tier	msp (all permissions)
Scope	No client_id scoping

License Keys (v1.20.0+)

Property	Value
Header	x-api-key
Key format	nvt_{tier}_{64 hex} (e.g., nvt_pro_a1b2..., nvt_biz_..., nvt_msp_...)
Tier prefix map	professional → pro, business → biz, msp → msp
Key prefix	First 14 chars (e.g., nvt_pro_a1b2c3), used for DB lookup
Hash	SHA-256 of full plaintext key, stored in license_keys.key_hash
Validation	Query by prefix, timing-safe hash comparison, check active/revoked/expired/limits
Client scoping	Upload auto-tags client_id from license key, overriding session JSON
Config file	templates/report_config.conf.example: NV_API_KEY (primary) or CONSULTATION_API_KEY (legacy alias)

Tier Permissions

Permission	Professional	Business	MSP
upload	Y	Y	Y
download	Y	Y	Y
sessions:read	Y	Y	Y
update:status	Y	Y	Y
benchmark		Y	Y
remediation:update		Y	Y
webhooks		Y	Y
compliance		Y	Y
agent:commands			Y
agent:groups			Y
db:health			Y
audit			Y
health:agents			Y

Default Limits

Tier	Max Sessions	Max Agents
Professional	50	5
Business	500	25
MSP	Unlimited	Unlimited

License Validation Endpoint (v4.1.0)

GET /api/license reports the status of the presented key without side effects. Implemented by license.mjs on the portal; consumed by lib/license_validate.sh and netvuln/license.py on the client.

Property	Value
Method	GET
Auth	x-api-key header (license key or legacy shared key)
URL derivation	${CONSULTATION_API_URL%/upload}/license (same convention the daemon uses for /health)

Request

GET /api/license
x-api-key: nvt_pro_a1b2c3...

Response (HTTP 200, even for revoked/expired/unknown keys)

{
  "valid": true,
  "tier": "professional",
  "client_id": "acme-corp",
  "active": true,
  "revoked": false,
  "expires_at": "2027-01-01T00:00:00Z",
  "limits": { "max_sessions": 50, "max_agents": 5 },
  "reason": "ok",
  "checked_at": "2026-06-09T12:00:00Z"
}

Field	Type	Notes
valid	boolean	True only when the key is usable right now
tier	string/null	professional / business / msp; the key's tier even when revoked/expired/inactive; null when the key is not recognized (not_found / invalid_format)
client_id	string/null	Client scoping tag bound to the key; null for unrecognized keys and the legacy shared key
active	boolean	Admin activation flag
revoked	boolean	Revocation flag
expires_at	string/null	ISO8601 expiry, null when perpetual
limits	object	max_sessions, max_agents (number/null)
reason	string	ok / revoked / expired / inactive / not_found / invalid_format
checked_at	string	Server timestamp of this check
legacy	boolean	true only for the legacy shared key; false for all nvt_ keys. Always present on the response.

Special cases:

Case	Response
Legacy shared key (NV_UPLOAD_API_KEY match)	{ "valid": true, "tier": "msp", "legacy": true }
Missing x-api-key header	HTTP 401 (the only non-200 auth outcome)
Server error / unreachable	5xx or no response; clients treat this as "can't tell", never as invalid

Client-Side Validation, Tier Gating, Cache and Grace

The scanner derives a tier from the key format (nvt_{pro|biz|msp}_{64 hex}) at load time and exports NV_LICENSED and NV_LICENSE_TIER. Key resolution checks names in the same order in bash and Python: NV_LICENSE_KEY, then NV_API_KEY, then CONSULTATION_API_KEY, then ~/.netvuln/license.key. Within a name, NV_API_KEY and CONSULTATION_API_KEY prefer a config-file value over the environment (bash config sourcing overwrites those variables; netvuln/license.py matches that behavior), while NV_LICENSE_KEY is never shadowed by a config and stays env-first. Non-nvt keys map to tier legacy (all features, deprecation warning); malformed nvt_ keys warn loudly and also map to legacy so a truncated paste never locks the client out. The server 403 stays authoritative.

Feature gates (nv_require_tier / netvuln.license.require_tier) are soft: warn and skip, never block core scanning. Minimum tiers: compliance, benchmark, remediation, webhooks require business; c2, agent_commands require msp; unknown features require msp.

Online validation (nv_license_refresh / netvuln.license.refresh) runs once per pipeline run and once per daemon heartbeat cycle:

• Results cache in ~/.netvuln/license_cache.json (mode 600, sha256 fingerprint prefix of the key, never the plaintext) with a 24h TTL.
• A fresh cache short-circuits the network call.
• valid:false downgrades the session to community features with a warning naming the reason.
• An unreachable portal keeps the cached tier for a further 72h grace window, then downgrades to community. A cached valid:false (revoked/expired/inactive) is authoritative regardless of age: the client stays downgraded until the portal answers valid:true again. Only when no fingerprint-matching cache exists at all does the format-derived tier stand (first run offline is not punished).
• A locally known past expires_at counts as expired regardless of connectivity.
• Hosts without jq skip the online/cache path entirely and keep the format/presence gate.

Shared Constants

Session ID Format

Pattern: YYYYMMDD_HHMMSS (e.g., 20260301_120000)

Generated by: nv_timestamp_filename() in lib/netvuln_common.sh

Used by: All Netlify Functions (stored as session_{id}/ prefix in Blobs)

Regex validation: /^[a-zA-Z0-9_-]+$/ (both repos)

Severity Levels

Canonical values: critical, high, medium, low, info

Generated by: classify_severity.sh in netvuln-tool

Consumed by: Collect portal filters, CSV export, PDF summary

Share URL Pattern

Format: https://bullium.com/share/?token={64-char-hex}

Token: 32 random bytes → 64 hex characters

Generated by: upload.mjs (server-side) and returned in upload response

Displayed by: push_session.sh (CLI output to user)

Remediation Status Values

Canonical values: new, persistent, in_progress, resolved, deferred

Generated by: remediation_tracker.sh (auto-detection), remediation_update.sh (manual)

Consumed by: Report template status badges, portal remediation dashboard

Remediation Tracking Fields

New fields added to .remediation[] entries:

Field	Type	Description
status	string	new, persistent, in_progress, resolved, deferred
status_changed	ISO 8601	Timestamp of last status change
note	string	Open-ended text annotation
first_seen	string	Session ID where this remediation first appeared
prior_status	string/null	Status from the prior session

New top-level key: .remediation_resolved[], an array of remediation entries from the prior session that are no longer detected in the current scan.

Remediation Summary (Metadata)

Added in v2.8.0. Stored in session metadata.json, populated at upload time by upload.mjs. Defaulted at read time by formatSession() in lib/format.mjs when the Neon column is null (the pre-Neon sessions.mjs blob backfill was removed in the v1.19.0 migration).

Field	Type	Description
remediation_summary.total	number	Count of active remediation items
remediation_summary.by_status.new	number	Items with status new
remediation_summary.by_status.persistent	number	Items with status persistent
remediation_summary.by_status.in_progress	number	Items with status in_progress
remediation_summary.by_status.resolved	number	Items with status resolved
remediation_summary.by_status.deferred	number	Items with status deferred
remediation_summary.resolved_count	number	Count of items in remediation_resolved[] (no longer detected)
remediation_summary.by_severity.critical	number	Items with severity critical
remediation_summary.by_severity.high	number	Items with severity high
remediation_summary.by_severity.medium	number	Items with severity medium
remediation_summary.by_severity.low	number	Items with severity low
remediation_summary.by_severity.info	number	Items with severity info

Exception / Accept-Liability Tracking (v3.1.0)

New fields on .remediation[] entries:

Field	Type	Description
exception_status	string|null	null (no exception), "accepted", "expired"
exception_reason	string|null	Justification for accepting risk
exception_approver	string|null	Who approved the exception
exception_approved_at	ISO 8601|null	Approval timestamp
exception_expiry	YYYY-MM-DD|null	Auto-expiry date
exception_expired	boolean	true if past expiry date

Dual Scoring Model (v3.1.0)

New fields in .summary:

Field	Type	Description
risk_score_operational	number	Post-exception score (0-100)
risk_grade_operational	string	Post-exception grade (A-F)
exception_impact.excepted_count	number	Active exception count
exception_impact.score_delta	number	Actual minus operational
exception_impact.excepted_findings	object	By-severity breakdown

Exception Summary (Portal Metadata, v3.1.0)

Extracted by upload.mjs, stored in metadata.json:

Field	Type	Description
exception_summary.total_exceptions	number	Total accepted + expired
exception_summary.by_status.accepted	number	Active exceptions
exception_summary.by_status.expired	number	Expired exceptions
exception_summary.score_delta	number	Score impact
risk_score_operational	number|null	Operational score
risk_grade_operational	string	Operational grade

Alerts Summary (Metadata)

Added in v2.9.0. Captured by nv_capture_alert_results() in alert_notifier.sh and written to session_results.json. Extracted at upload time by upload.mjs into metadata. Defaulted at read time by formatSession() in lib/format.mjs when the Neon column is null (the pre-Neon sessions.mjs blob backfill was removed in the v1.19.0 migration).

Field	Type	Description
alerts_summary.evaluated_at	ISO 8601 / null	When alerts were evaluated
alerts_summary.total_triggered	number	Count of alerts fired (0 if none)
alerts_summary.alerts_triggered	array	List of triggered alert objects
alerts_summary.alerts_triggered[].type	string	risk_threshold, new_critical, scan_complete
alerts_summary.alerts_triggered[].timestamp	ISO 8601	When the alert fired
alerts_summary.alerts_triggered[].message	string	Human-readable alert description
alerts_summary.alerts_triggered[].channels_notified	array	Channels used (e.g., ["email","slack"])
alerts_summary.config	object	Alert config snapshot
alerts_summary.config.risk_threshold	number	Configured risk threshold
alerts_summary.config.on_new_critical	boolean	Whether new_critical alerts enabled
alerts_summary.config.on_scan_complete	boolean	Whether scan_complete alerts enabled
alerts_summary.config.channels	array	Configured channel names

Executive Summary

Added in v2.9.0. The exec_summary_html multipart field is stored as session_{id}/exec_summary.html in blob storage. Presence is tracked via has_exec_summary (boolean) in session metadata, defaulted at read time by formatSession() in lib/format.mjs when the Neon column is null.

CLI sources: push_session.sh (-e flag or auto-detect exec_summary_*.html), upload_session.sh (-e flag or auto-detect), submit_session.sh (6th parameter).

Portal: Download via GET /api/download?session={id}&file=exec_summary.html&mode=view.

Benchmark Data

Benchmark percentile data is generated client-side by benchmark_compare.sh and stored in session_results.json under the .benchmark key. The portal reads this on-demand when the user expands a session row.

Field	Type	Description
benchmark.sample_size	number	Sessions in comparison cohort
benchmark.fetched_at	ISO 8601	When benchmark was computed
benchmark.metrics.risk_score	object	Risk score percentile data
benchmark.metrics.findings_per_host	object	Findings/host percentile data
benchmark.metrics.critical_high_ratio	object	Critical+High % percentile data
benchmark.metrics.*.value	number	Raw metric value for this session
benchmark.metrics.*.percentile	number	Percentile rank (0-100, higher = better)
benchmark.metrics.*.label	string	Human-readable metric name
benchmark.metrics.*.direction	string	lower_is_better

Scheduling Metadata

Added in v2.10.0. Injected into session_results.json by scheduled_scan.sh before auto-upload. Extracted at upload time by upload.mjs into metadata. Backfilled on first read by sessions.mjs (reads session_results.json from blob store).

Field	Type	Description
metadata.schedule_id	string	Deterministic hash: sched_<12-char-md5> of target|cron
metadata.schedule_cron	string	Cron expression (e.g., 0 2 * * 0)
metadata.schedule_profile	string	Scan profile name (e.g., full_recon)
metadata.is_scheduled	boolean	true for automated scans

CLI source: scheduled_scan.sh injects via _inject_schedule_metadata() after scan; netvuln_daemon.sh injects via _nvd_inject_schedule_metadata() with scheduled_by=daemon

Additional daemon-injected field:

Field	Type	Description
metadata.scheduled_by	string	"daemon" when run by daemon, "cron" when run by cron

Portal: Schedule dashboard groups sessions by schedule_id, displays cron expression, run count, and risk score timeline per schedule.

Session License Metadata (v4.2.3)

Added in netvuln-tool v4.2.3 (#46). nv_json_set_license() writes a metadata.license block into every session JSON (called from run_pipeline.sh):

Field	Type	Description
metadata.license.tier	string	Resolved license tier (pro / biz / msp / legacy)
metadata.license.fingerprint	string	SHA-256 fingerprint of the key (never the plaintext key)
metadata.license.source	string	Where the key was resolved from (env var name or config path)
metadata.license.validated_at	ISO 8601	When the key was last validated

Portal: currently ignores this block (upload.mjs does not read metadata.license). Documented here so a future portal feature does not collide with or duplicate it. The fingerprint is safe to store; the plaintext key is never written to the session JSON.

Daemon Heartbeat Payload

Added in v3.0.0. The daemon (netvuln_daemon.sh) sends periodic JSON heartbeats to the portal health endpoint. Heartbeat interval is configurable via DAEMON_HEARTBEAT_INTERVAL.

Property	Value
URL	POST /api/health (derived from CONSULTATION_API_URL)
Auth	x-api-key header
Interval	DAEMON_HEARTBEAT_INTERVAL seconds (default: 300)

Heartbeat Payload

Field	Type	Description
agent_id	string	nvd_<8-char-md5> of hostname
version	string	netvuln-tool version
timestamp	ISO 8601	When heartbeat was sent
status	string	"running"
uptime_seconds	number	Seconds since daemon start
client_id	string	Client identifier from first loaded config
schedules	array	Per-schedule status objects
schedules[].config	string	Config filename
schedules[].schedule_id	string	Deterministic schedule hash
schedules[].last_run	ISO 8601/null	Last scan execution time
schedules[].last_status	string	none, success, failed
schedules[].next_run	ISO 8601	Next scheduled scan time
schedules[].retry_count	number	Consecutive failure count
system_info	object	Optional host telemetry (OS, packages, network, config) gathered by the daemon; absent on older daemons. Stored in the system_info jsonb column.

Client ID / Multi-Tenancy

Added in v2.14.0. Optional client identifier for portal multi-tenancy session scoping. Scripts-side injection shipped in v2.14.0; portal-side filtering shipped in v2.14.1 (bullium-site v1.8.0).

Field	Type	Description
metadata.client_id	string	Client identifier for session scoping (e.g., acme-corp)

Config variable: CLIENT_ID in report_config.conf.example

CLI source: run_pipeline.sh passes CLIENT_ID as 5th arg to nv_json_set_client(), or calls nv_json_set_client_id() directly when CLIENT_NAME is empty but CLIENT_ID is set.

Portal (shipped v2.14.1 / bullium-site v1.8.0): getUserScope() in auth.mjs extracts app_metadata.roles and app_metadata.client_ids; sessions.mjs filters by scope; download.mjs, update.mjs, delete.mjs enforce 403 for out-of-scope access. Backward compatible: no app_metadata → admin (sees all sessions).

Report White-Label Configuration

Added in v2.14.0. Config-driven branding variables that affect report HTML output via CSS custom property overrides in rt_brand_overrides().

Config Variable	CSS Custom Property	Default	Description
REPORT_PRIMARY_COLOR	--primary-color	#dc3545	Primary accent color (links, borders, badges)
REPORT_HEADER_BG	--header-bg	#1a1a2e	Report header background
REPORT_HEADER_TEXT	--header-text	#ffffff	Report header text color
REPORT_SUBTITLE	-	Bullium Consulting LLC	Header subtitle text
REPORT_FOOTER_TEXT	-	Generated by netvuln-tool...	Footer paragraph text
REPORT_FOOTER_EMAIL	-	info@bullium.com	Footer contact email
REPORT_FOOTER_URL	-	https://bullium.com	Footer website URL
COMPANY_LOGO	-	Auto-detected	Path to custom logo file for report header

CLI source: Variables read from config file, applied by generate_report.sh calling rt_brand_overrides after rt_html_head.

Portal impact: White-labeled reports display with custom branding when viewed via the share page or downloaded from the collect portal. No portal code changes needed.

Health Check Endpoint

Added in v1.7.0. Extended in v1.9.0 with daemon heartbeat support.

GET /api/health (public)

Property	Value
URL	GET /api/health
Auth	None (public)

Response: { "status": "ok", "version": "1.25.9", "timestamp": "ISO 8601", "environment": "production" }

version is the portal package version (illustrative here) and environment reflects the Netlify deploy context (production / deploy-preview / branch-deploy / dev, or unknown when unset), so it is not always production.

GET /api/health?agents=true (protected)

Property	Value
URL	GET /api/health?agents=true
Auth	Netlify Identity JWT
Scoping	Non-admin users see only agents matching their client_ids

Response: { "agents": [...], "stale_threshold_seconds": 600 }

Each agent object includes agent_id, version, timestamp, status, uptime_seconds, client_id, schedules, system_info, received_at, is_stale (computed), age_seconds (computed).

POST /api/health (daemon heartbeat)

Property	Value
URL	POST /api/health
Auth	x-api-key header
Storage	agent_heartbeats table in Neon PostgreSQL (upsert on heartbeat)
Stale threshold	600 seconds (2x default heartbeat interval)

Request body: See Daemon Heartbeat Payload section.

Response: { "success": true, "received_at": "ISO 8601", "commands": [ ... ] }

The heartbeat response piggybacks any pending C2 commands for the agent (up to 20), marking them sent in the same cycle. Each command object matches the GET /api/agent-commands agent-poll shape: id, command_type, payload, priority, created_by, created_at. The commands array is [] when nothing is queued.

Agent Lifecycle Endpoint (PATCH /api/agents)

Admin agent management (portal-side). Deregister removes a daemon agent from the active roster (and its group memberships); reenable restores it; purge permanently deletes a deregistered agent and its history. Implemented by agents.mjs.

Property	Value
URL	PATCH /api/agents
Auth	Netlify Identity JWT (admin-only)
Storage	agent_heartbeats (+ agent_commands, command_results, agent_groups on purge); emits an audit event + webhook

Request body:

{ "agent_id": "nvd_abc12345", "action": "deregister" }

action is "deregister", "reenable", or "purge".

Response	Meaning
200 { "ok": true }	Action applied
404 Agent not found	Unknown agent_id
409	Already in the requested state, or purge attempted on an agent that is not deregistered

Deregister sets deregistered_at, drops the agent's group memberships, and emits the agent_deregistered audit event + webhook; reenable clears deregistered_at and emits agent_reregistered. A deregistered agent's heartbeat (POST /api/health) returns 403 "Agent unregistered..." until it is reenabled.

Purge (v1.26.0) permanently deletes an already-deregistered agent: it removes the agent_heartbeats row plus the agent's agent_commands (cascading command_results) and agent_groups rows. audit_events are retained and an agent_purged event + webhook are emitted. Purging an agent that is not deregistered returns 409 (deregister it first).

Synthetic Collection Generator (synthetic_collection.sh)

Creates test session datasets for development, demos, and CI/CD validation.

Flag	Default	Description
--target	(required)	Target string
--sessions	1	Number of sessions to generate
--findings	15	Findings per session
--mix	"critical:10,..."	Severity distribution (%)
--trend	stable	improving|worsening|stable
--with-exceptions	0	Add N exception entries
--seed	(random)	Deterministic seed
--client-id	(empty)	Portal multi-tenancy

Report HTML Coupling

The share page (~/bullium-site/share/index.html) injects a bridge script into the report HTML. These CSS/HTML markers in report_template.sh must be preserved for the bridge to work:

Marker	Type	Purpose
#consultation-section	ID	Anchor link target for CTA buttons
.consultation-section	Class	Section container styling
#scheduleBtn	ID	Schedule consultation button
</body>	HTML tag	Bridge script injection point

CRITICAL: Use lastIndexOf('</body>') for injection, never String.replace('</body>', ...) because the report's loadScheduleWidget() function contains </body> inside a JS string literal.

Alert Webhook Payload

The alert_notifier.sh script dispatches structured JSON alerts to a configured webhook URL (ALERT_WEBHOOK_URL).

Field	Type	Description
event	string	Always "netvuln_alert"
alert_type	string	risk_threshold, new_critical, scan_complete, test
timestamp	ISO 8601	When the alert was generated
session_id	string	Session identifier (e.g., 20260301_120000)
target	string	Scan target(s)
risk_score	number	Current risk score (0-100)
risk_grade	string	Letter grade (A-F)
new_critical_high	number	Count of new critical/high findings vs prior scan
total_findings	number	Total findings count
subject	string	Alert subject line
message	string	Alert body text

Alert Types

Type	Condition	Config
risk_threshold	risk_score >= ALERT_RISK_THRESHOLD	ALERT_RISK_THRESHOLD (default: 60)
new_critical	New Critical/High findings vs prior scan	ALERT_ON_NEW_CRITICAL (default: true)
scan_complete	Any scan completion	ALERT_ON_SCAN_COMPLETE (default: false)
test	Manual test via --test flag	-

Channels

Channel	Config Variable	Format
Email	ALERT_EMAIL (fallback: REMINDER_EMAIL)	Plain text
Slack	ALERT_SLACK_WEBHOOK	Slack Block Kit JSON
Webhook	ALERT_WEBHOOK_URL	JSON (see payload above)

Compliance Mapping (v3.2.0)

.compliance[] Array

Field	Type	Description
framework	string	Framework name (CIS Controls v8, NIST CSF, PCI-DSS v4.0, SOC 2, ORC 9.64)
control_id	string	Control identifier (e.g., "CIS 3.10", "PR.DS-2", "9.64(C)(1)")
control_name	string	Human-readable control name
finding_ids	string[]	Finding IDs mapped to this control

.summary.compliance_summary

Field	Type	Description
total_mappings	number	Total compliance control mappings
by_framework[].framework	string	Framework name
by_framework[].mapped_controls	number	Controls with associated findings
by_framework[].total_controls	number	Total controls checked
by_framework[].gap_controls	number	Controls without findings
by_framework[].findings_count	number	Findings mapped to this framework

Portal Metadata

Field	Source	Description
compliance_summary	summary.compliance_summary	Compliance summary object
has_topology	topology.nodes.length > 0	Boolean topology flag

ORC 9.64 Readiness (v4.1.0)

Ohio Revised Code 9.64 requires every Ohio political subdivision to adopt a cybersecurity program. compliance_mapper.sh maps scan findings to the five technical divisions, ORC 9.64(C)(1) through ORC 9.64(C)(5), and records four attestation items a scanner cannot observe (annual training, incident notifications, ransomware payment policy).

Activation: opt-in via ORC_964_ENABLED="true" or CLIENT_TYPE="political_subdivision". The feature is gated behind the compliance license permission (business tier or higher); without it the mapper strips ORC rows and omits .orc_964 entirely.

Gap math: total_controls for ORC is 5 (technical divisions only) so gap analysis never penalizes subdivisions for policy items a scanner cannot see.

.orc_964 Object

Field	Type	Description
enabled	boolean	Always true when the block is present
deadlines.county_city	string	"2026-01-01" (counties and cities)
deadlines.other	string	"2026-07-01" (all other subdivisions)
deadlines.applicable	string	county_city or other, from ORC_SUBDIVISION_CLASS (default other)
divisions[]	object[]	Always all five technical divisions ORC 9.64(C)(1) through (C)(5)
divisions[].id	string	Division id, e.g. "9.64(C)(1)"
divisions[].name	string	Division requirement text
divisions[].type	string	Always "technical"
divisions[].mapped	boolean	True when at least one finding maps to the division
divisions[].finding_ids	string[]	Finding IDs evidencing the division (empty when unmapped)
divisions[].finding_count	number	finding_ids length
attestations[]	object[]	Four items: 9.64(C)(6) training, 9.64(D)(1) 7-day Ohio Homeland Security notice, 9.64(D)(2) 30-day Auditor of State notice, 9.64(B) ransomware payment resolution
attestations[].id	string	Statutory reference
attestations[].name	string	Requirement text
attestations[].state	string	true, false, or unknown (from ORC_ATTEST_* config, default unknown)
attestations[].note	string	Optional guidance note (training item only)
coverage.technical_divisions_total	number	Always 5
coverage.technical_divisions_covered	number	Distinct ORC divisions present in .compliance

Portal Metadata

Field	Source	Description
orc964_summary.enabled	orc_964.enabled	Boolean readiness flag
orc964_summary.technical_covered	orc_964.coverage.technical_divisions_covered	Divisions evidenced by findings
orc964_summary.technical_total	orc_964.coverage.technical_divisions_total	Always 5
orc964_summary.attest_pending	count of orc_964.attestations[] with state != "true"	Attestation items still requiring confirmation

Network Topology (v3.2.0)

.topology.nodes[]

Field	Type	Description
ip	string	Host IP address
hostname	string	Hostname if available
os_guess	string	OS detection guess
port_count	number	Number of open ports
risk_score	number	Per-host risk score (0-100)
risk_grade	string	Risk grade (A-F)
subnet	string	/24 subnet prefix

.topology.links[]

Field	Type	Description
source_ip	string	Source host IP
target_ip	string	Target host IP
type	string	Link type ("subnet")

.topology.subnets[]

Field	Type	Description
prefix	string	Subnet prefix (e.g., "10.0.0")
host_count	number	Hosts in subnet
avg_risk	number	Average risk score

Database Health Endpoint (v1.19.0)

Admin-only endpoint for monitoring Neon PostgreSQL database health.

Detail	Value
URL	GET /api/db-health
Auth	Netlify Identity JWT (admin-only)
Storage	Queries Neon PostgreSQL system tables

Response:

{
  "table_counts": { "sessions": 42, "session_notes": 108, "status_history": 75, "share_tokens": 40, "agent_heartbeats": 3, "agents": 3, "webhook_subscriptions": 2, "webhooks": 2, "audit_events": 310 },
  "database_info": { "size_bytes": 52428800, "size_human": "50.00 MB", "active_connections": 5, "host": "...", "version": "PostgreSQL 17.8", "database_name": "neondb", "max_connections": "100", "provider": "Neon PostgreSQL" },
  "recent_activity": [{ "event_type": "session_uploaded", "session_id": "20260301_120000", "actor": "admin@bullium.com", "data": {}, "created_at": "ISO 8601", "timestamp": "ISO 8601" }],
  "session_stats": { "by_status": { "new": 15, "in_review": 10, "resolved": 12, "archived": 5 }, "by_client": [{ "client_id": "acme-corp", "client": "Acme Corp", "count": 20 }], "recent_uploads": [{ "session_id": "...", "client": "...", "target": "...", "risk_score": 65, "risk_grade": "D", "uploaded_at": "ISO 8601" }] },
  "neon_usage": { "tier": "Free", "storage_limit_mb": 512, "storage_used_mb": 50, "storage_mb": 50, "storage_pct": 9.8, "storage_percent": 9.8, "active_connections": 5, "health": "green" }
}

Note: DB size uses pg_column_size() row sampling (Neon separates compute from storage, so pg_database_size returns 0). Version and max_connections fall back to SHOW commands when the Neon pooler restricts standard functions.

Remediation Update Endpoint (v1.20.0)

Update individual remediation item statuses from the report view.

Detail	Value
URL	PATCH /api/remediation-update
Auth	Netlify Identity JWT (scoped by client_id)
Storage	Netlify Blobs (read-modify-write session_results.json) + Neon PostgreSQL

Request body:

{
  "session_id": "20240301_120000",
  "items": [
    { "title": "Close SMTP Open Relay", "status": "resolved" },
    { "title": "Add Missing Security Headers", "status": "new" }
  ]
}

Response:

{
  "success": true,
  "session_id": "20240301_120000",
  "changed": 2,
  "remediation_summary": { "total": 10, "by_status": { ... }, "resolved_count": 5 }
}

Items are matched by title. Status resolved moves items from remediation[] to remediation_resolved[]; other statuses move them back or update in-place. An audit_events entry with type remediation_updated is logged. Uses version-based optimistic locking on blob writes (max 3 retries); returns 409 Conflict if concurrent updates exhaust retries.

Audit Log Endpoint (v1.20.0)

Tenant-scoped audit event log with time-based retention.

Detail	Value
URL	GET /api/audit
Auth	Netlify Identity JWT (admins see all; scoped users see own client_id events)
Storage	Neon PostgreSQL audit_events table

Query parameters:

Param	Type	Default	Description
days	int	30	Time window (1-90)
type	string	-	Filter by event type (scan_completed, status_changed, remediation_updated, session_deleted, agent_first_seen, agent_deregistered, agent_reregistered, agent_purged, license_created, license_revoked, license_deleted)
session	string	-	Filter by session ID

Response:

{
  "events": [
    {
      "id": 1,
      "event_type": "status_changed",
      "session_id": "20240301_120000",
      "client_id": "acme-corp",
      "actor": "user@acme.com",
      "data": { "previous_status": "new", "new_status": "in_review" },
      "created_at": "2026-03-19T12:00:00Z"
    }
  ],
  "days": 30,
  "total": 1
}

Fire-and-forget cleanup prunes events older than 90 days on each request.

Webhook Events (v1.12.0 / Portal API v2)

Added in bullium-site v1.12.0. Structured event system for PSA/RMM integrations. Replaces threshold-based alert_notifier.sh webhooks with subscription-based event delivery, HMAC signing, and multi-subscriber dispatch.

Note: This system is separate from alert_notifier.sh alerts. The legacy alert webhook (ALERT_WEBHOOK_URL) continues to work as-is for scan-time threshold alerts. The portal webhook events fire on data changes (upload, status change, deletion).

Subscription Management

Property	Value
URL	/api/webhooks
Auth	Netlify Identity JWT (admin-only)
Methods	GET (list), POST (create), PATCH (update), DELETE (remove)
Storage	webhook_subscriptions table in Neon PostgreSQL

Subscription schema:

Field	Type	Description
id	string	whsub_<12-char-hex>
url	string	HTTPS webhook endpoint (no localhost/private IPs)
event_types	string[]	Subscribed event types
signing_secret	string	64-char hex HMAC key (shown once at creation)
enabled	boolean	Whether subscription receives events
description	string	Human-readable label
client_id	string	Optional, scopes to matching sessions
created_by	string	Admin email
created_at	ISO 8601	Creation timestamp
updated_at	ISO 8601	Last modification timestamp

Event Payload Format

{
    "api_version": "2",
    "event_type": "scan_completed",
    "event_id": "evt_<16-char-hex>",
    "timestamp": "ISO 8601",
    "data": { }
}

Delivery Headers

Header	Value
Content-Type	application/json
X-NV-Event	Event type name
X-NV-Signature-256	sha256=<HMAC-SHA256 of body using signing_secret>
X-NV-Delivery	Event ID
X-NV-API-Version	2
User-Agent	Bullium-netvuln-webhook/2.0

Event Types

Event	Source	Trigger
scan_completed	upload.mjs	After session blob writes succeed
status_changed	update.mjs	After status transition metadata write
session_deleted	delete.mjs	After all session blobs deleted
report_expiring	sessions.mjs	Lazy-evaluated during session listing when a session enters the 7-day expiry window; expiry_notified_at prevents duplicates
agent_deregistered	agents.mjs	After an admin deregisters a daemon agent
agent_reregistered	agents.mjs	After an admin reenables a deregistered agent
agent_purged	agents.mjs	After an admin permanently deletes a deregistered agent

Event Data Fields

scan_completed: session_id, client, client_id, target, risk_score, risk_grade, total_findings, total_hosts, severity_summary, share_url, uploaded_at, tool_version

status_changed: session_id, client, client_id, target, previous_status, new_status, changed_by, changed_at

session_deleted: session_id, client, client_id, deleted_by, deleted_at

report_expiring: session_id, client, client_id, target, uploaded_at, expires_at, days_remaining

agent_deregistered: agent_id, deregistered_by

agent_reregistered: agent_id, reregistered_by

agent_purged: agent_id, purged_by

Delivery Semantics

• Fire-and-forget with single retry (5s timeout, 500ms retry delay)
• Promise.allSettled() for parallel delivery to multiple subscribers
• Non-fatal: dispatch errors never fail the triggering operation
• HTTPS-only webhook URLs enforced at subscription creation

C2 Command Pipeline (v1.20.0)

Remote command-and-control for netvuln-tool daemon agents. Commands are queued by portal admins and polled by agents via heartbeat cycle.

POST /api/agent-commands (admin: queue command)

Property	Value
URL	POST /api/agent-commands
Auth	Netlify Identity JWT (admin-only)
Storage	agent_commands table in Neon PostgreSQL

Request body:

{
  "command_type": "run_scan",
  "agent_id": "nvd_abc12345",
  "payload": { "profile": "full_recon", "targets": "192.168.1.0/24", "upload": true },
  "priority": 0
}

run_scan payload keys: targets (required), profile (optional; full_recon/full → --full, quick/quick_recon → -P quick_recon, other → default phases), and upload (optional boolean; when true, the resulting session is pushed to the portal after a successful scan). push_session payload: optional session_id (defaults to latest).

For group broadcast, replace agent_id with group_target:

{
  "command_type": "restart",
  "group_target": "prod-scanners",
  "payload": {},
  "priority": 5
}

Valid command types: run_scan, update_schedule, push_config, push_session, restart, shutdown, update, set_log_level

Response (HTTP 201):

{
  "success": true,
  "commands": [{ "id": 1, "agent_id": "nvd_abc12345", "command_type": "run_scan", "status": "pending", "created_at": "ISO 8601" }]
}

GET /api/agent-commands (agent polling)

Property	Value
URL	GET /api/agent-commands?agent_id=nvd_abc12345
Auth	x-api-key header
Side effect	Marks returned commands as sent

Response: { "commands": [{ "id": 1, "command_type": "run_scan", "payload": {...}, "priority": 0, "created_by": "admin@bullium.com", "group_target": null, "created_at": "ISO 8601" }] }

group_target is the target group name for broadcast commands and null for direct-to-agent commands; it is present in both the agent-poll and admin-history responses.

Dispatch ordering. Pending commands are selected ORDER BY priority DESC, created_at ASC in both the agent poll and the heartbeat piggyback (health.mjs). Higher priority (0 to 10) is dispatched first within a batch, with oldest-first as the tiebreak. The poll returns up to 50 commands; the heartbeat piggyback up to 20. Priority is a portal-side ordering hint only: the agent does not read priority and executes commands in the order received.

GET /api/agent-commands (admin history)

Property	Value
URL	GET /api/agent-commands?limit=100 or GET /api/agent-commands?command_id=42
Auth	Netlify Identity JWT (admin-only)
Side effect	None (read-only)

List mode returns { "commands": [...] }. Detail mode (?command_id=X) returns { "command": {...}, "result": { "exit_code": 0, "stdout": "...", "stderr": "...", "session_id": "20260714_120000", "completed_at": "ISO 8601" } }. session_id is populated for run_scan and push_session results and links to the collect-portal session detail (/collect/?session=<id>); it is null for other commands.

PATCH /api/agent-commands (agent status update)

Property	Value
URL	PATCH /api/agent-commands
Auth	x-api-key header
Storage	Updates agent_commands status; inserts command_results on completion/failure

Request body:

{
  "command_id": 1,
  "agent_id": "nvd_abc12345",
  "status": "completed",
  "exit_code": 0,
  "stdout": "Scan complete. 5 hosts found.",
  "stderr": "",
  "session_id": "20260714_120000"
}

session_id is optional; the agent includes it for run_scan and push_session so the portal can store it on command_results and link History to the session detail.

Valid statuses: pending → sent → acked → completed | failed

Agent Groups

Property	Value
URL	/api/agent-groups
Auth	Netlify Identity JWT (admin-only)
Methods	GET (list groups), POST (add agent to group), DELETE (remove agent from group)

POST body: { "agent_id": "nvd_abc12345", "group_name": "prod-scanners" }

DELETE params: ?agent_id=nvd_abc12345&group_name=prod-scanners

GET response: { "groups": { "prod-scanners": [{ "agent_id": "nvd_abc12345", "created_at": "ISO 8601" }] } }

C2 Webhook Events

Event	Source	Trigger
command_dispatched	agent-commands.mjs	After command(s) inserted into queue
command_completed	agent-commands.mjs	Agent reports successful execution
command_failed	agent-commands.mjs	Agent reports failed execution

C2 Audit Events

All commands are logged to audit_events: command_dispatched (actor: admin email), command_completed (actor: agent_id), command_failed (actor: agent_id).

Cross-Project Change Checklist

When modifying shared interfaces, check BOTH repositories:

• [ ] Upload fields → submit_session.sh AND upload.mjs
• [ ] Severity levels → classify_severity.sh AND collect portal filter dropdown
• [ ] Session ID format → nv_timestamp_filename() AND all Netlify Functions
• [ ] Report HTML structure → report_template.sh AND share/index.html bridge
• [ ] API response format → submit_session.sh response parsing AND upload.mjs
• [ ] Share URL pattern → push_session.sh display AND share.mjs lookup
• [ ] Config variables → report_config.conf.example AND upload.mjs env vars
• [ ] Remediation status → remediation_tracker.sh AND collect portal remediation view
• [ ] Alert webhook payload → alert_notifier.sh AND consumer webhook handler
• [ ] Remediation summary → upload.mjs metadata extraction AND collect portal stats/progress bars
• [ ] Alerts summary → alert_notifier.sh capture AND upload.mjs extraction AND collect portal badges
• [ ] Exec summary upload → submit_session.sh 6th param AND upload.mjs AND download.mjs whitelist
• [ ] Benchmark data → benchmark_compare.sh JSON key AND collect portal benchmark section
• [ ] Schedule metadata → scheduled_scan.sh injection AND upload.mjs extraction AND collect portal schedule dashboard
• [ ] Client ID scoping → run_pipeline.sh / nv_json_set_client_id() AND upload.mjs extraction AND collect portal client filtering (v2.14.1)
• [ ] Report white-label → report_config.conf.example variables AND report_template.sh CSS overrides (portal-transparent, no portal changes needed)
• [ ] Daemon heartbeat → netvuln_daemon.sh JSON POST AND portal /api/health endpoint (v3.0.0)
• [ ] Daemon schedule metadata → netvuln_daemon.sh scheduled_by=daemon injection AND upload.mjs extraction (v3.0.0)
• [ ] Exception status → remediation_update.sh CLI AND remediation_tracker.sh carry-forward AND collect portal exception view
• [ ] Exception summary → upload.mjs extraction AND collect portal Risk Register + badges
• [ ] Operational score → risk_score.sh dual model AND upload.mjs extraction AND collect portal dual display
• [ ] Compliance mapping → compliance_mapper.sh AND upload.mjs extraction AND collect portal compliance panel
• [ ] Topology data → network_topology.sh AND upload.mjs extraction AND collect portal topology panel
• [ ] Webhook event types → webhook-dispatch.mjs VALID_EVENT_TYPES AND webhooks.mjs validation AND API_CONTRACT.md event type table
• [ ] Webhook event payload → webhook-dispatch.mjs payload builder AND consumer verification AND API_CONTRACT.md payload schema
• [ ] Webhook subscription schema → webhooks.mjs CRUD AND API_CONTRACT.md subscription model
• [ ] Report refresh → refresh_reports.sh AND refresh.mjs endpoint AND API_CONTRACT.md refresh endpoint
• [ ] License key format → nvt_{tier}_{hex} generation in licenses.mjs AND auth.mjs validation AND CLI report_config.conf NV_API_KEY variable (or legacy CONSULTATION_API_KEY)
• [ ] E2E license tests → tests/e2e/test_license_lifecycle.sh AND auth.mjs permission matrix AND licenses.mjs CRUD
• [ ] License tier permissions → TIER_PERMISSIONS in auth.mjs AND requirePermission() in API-key endpoints AND admin dashboard Licenses panel AND API_CONTRACT.md tier table
• [ ] License validation → license.mjs + describeLicense AND lib/license_validate.sh AND netvuln/license.py AND e2e suite_validate
• [ ] ORC 9.64 readiness → compliance_mapper.sh .orc_964 AND upload.mjs extraction AND schema.mjs orc964Summary AND collect portal ORC readiness panel
• [ ] C2 command priority → agent-commands.mjs + health.mjs ORDER BY priority DESC AND admin dashboard priority input (0 to 10) AND API_CONTRACT.md dispatch-ordering note AND C2_GUIDE.md section 4
• [ ] C2 config persistence → lib/command_handler.sh .c2_overrides.conf write AND netvuln_daemon.sh _nvd_load_configs re-apply AND admin dashboard Runtime Config editor AND C2_GUIDE.md section 6
• [ ] Agent purge → agents.mjs action: "purge" (cascade + agent_purged event) AND admin dashboard "Remove permanently" AND API_CONTRACT.md Agent Lifecycle Endpoint
• [ ] C2 command types → lib/command_handler.sh dispatch switch AND agent-commands.mjs valid types AND admin dashboard C2_COMMAND_SPECS builder AND API_CONTRACT.md valid-command-types list (push_session added in #86)
• [ ] C2 run_scan profile mapping → lib/command_handler.sh (quick/quick_recon → -P quick_recon, full/full_recon → --full) AND vulnscan_recon.sh accepted flags AND admin dashboard run_scan profile options AND C2_GUIDE.md section 5
• [ ] C2 result session_id → lib/command_handler.sh status PATCH session_id AND agent-commands.mjs PATCH store + GET return AND schema.mjs command_results.session_id column AND admin dashboard History Session ID column linking /collect/?session=<id>
• [ ] C2 session push → lib/command_handler.sh push_session + run_scan upload flag AND admin dashboard builder AND collect portal ?session=<id> deep-link

Secrets & Environment Management

Full credential inventory, environment setup, rotation procedures, and security practices are documented in Secrets Management.

Key topics covered:

• Complete secret inventory for both repos (what, where, risk level)
• Dev vs production credential separation (Neon branching, two-key rule)
• Environment setup guides (local dev, Netlify dashboard, scanner configs)
• Rotation procedures for each secret type (license keys, DB URLs, API tokens)
• Secret hygiene checklist and quarterly audit commands
• Architecture rationale for current approach vs vault

Deployment Order

When a change touches both repositories, deploy in the correct order to avoid breaking the integration. The general rule: the consumer deploys before the producer for new fields, and the producer deploys before the consumer for removed fields.

Adding a New Feature (Field, Endpoint, or Payload)

1. bullium-site: Add support for the new field/endpoint (accept but don't require)
2. Merge to dev → validate on dev Neon branch
3. Merge dev → main (production portal now accepts the new field)
4. netvuln-tool: Start sending the new field
5. Release netvuln-tool version

Why portal first: If the scanner sends a field the portal doesn't expect, upload may fail or silently drop data. If the portal accepts a field the scanner doesn't send yet, it gracefully handles the absence (null/default).

License validation: the portal license.mjs (GET /api/license) deploys to production BEFORE any netvuln client release that performs online validation. Clients treat an unreachable or missing endpoint as "can't tell" and fall back to cache/format tiers, so the old-client/new-portal combination is safe, but new clients pointed at an old portal would burn their grace window.

Removing a Deprecated Feature

1. netvuln-tool: Stop sending the deprecated field
2. Release netvuln-tool version
3. Wait for all deployed scanners to update (or set a deprecation deadline)
4. bullium-site: Remove support for the deprecated field
5. Merge to dev → validate → merge dev → main

Why scanner first: If the portal stops accepting a field that scanners still send, existing deployments break. Remove the source before the sink.

Schema-Only Changes (No Scanner Impact)

1. Modify schema.mjs on feature branch
2. Merge to dev → Netlify branch deploy runs drizzle-kit push against dev Neon
3. Verify functions work on dev deploy (test all affected endpoints)
4. Merge dev → main → drizzle-kit push against production Neon

Breaking Changes (Rare, avoid if possible)

If a change is unavoidably breaking (e.g., renaming a field, changing a response format), coordinate a cutover:

1. bullium-site: Add support for BOTH old and new format (backward compatible)
2. Deploy portal to production
3. netvuln-tool: Switch to new format
4. Release netvuln-tool version with minimum portal version requirement
5. After all scanners update: remove old format support from portal

Version Compatibility Matrix

Minimum portal version required for each scanner feature set.

netvuln-tool Version	Minimum bullium-site	Feature Introduced
v2.6.2	v1.4.0	Session upload, share page, collection portal
v2.8.0	v1.5.0	Risk scoring, remediation tracking, alert webhooks
v2.9.0	v1.6.0	Executive summary upload, alerts_summary metadata
v2.10.0	v1.7.0	Schedule metadata, health check endpoint
v2.14.0	v1.8.0	CLIENT_ID multi-tenancy, report white-labeling
v3.0.0	v1.9.0	Daemon heartbeat, agent status dashboard
v3.1.0	v1.10.0	Exception/accept-liability, dual scoring model
v3.2.0	v1.11.0	Compliance mapping, network topology
v3.2.1+	v1.12.0	Webhook event system (portal-side only)
v3.2.1+	v1.19.0	Neon PostgreSQL (replaces blob metadata)
v3.2.1+	v1.20.0	License keys, C2 command pipeline
v4.1.0	v1.24.0	ORC 9.64 framework + readiness view (business tier, compliance permission); GET /api/license scan-time validation
v4.2.3	v1.20.0	Session metadata.license block (fingerprint only; portal-transparent, currently ignored)

Backward compatibility rule: The portal always accepts older scanner formats. A v2.6.2 scanner can still upload to a v1.21.1 portal. Newer metadata fields simply won't be populated. However, a v3.2.0 scanner uploading compliance data to a v1.8.0 portal would lose that data silently (portal doesn't extract it).

Architecture Complexity Guide

As netvuln-tool grows beyond a Bash script collection into a multi-subsystem platform (Python orchestrator, persistent daemon, C2 pipeline, plugin system), this section documents known integration points, friction areas, and the plan for managing complexity.

Current Subsystem Map

┌─────────────────────────────────────────────────────────┐
│                    netvuln-tool                          │
│                                                         │
│  ┌──────────────┐   ┌──────────────┐   ┌────────────┐  │
│  │ Python CLI   │   │ Bash Daemon  │   │ Plugin     │  │
│  │ (netvuln/)   │   │ (netvuln_    │   │ Loader     │  │
│  │              │   │  daemon.sh)  │   │ (plugin_   │  │
│  │ recon cmd ✓  │   │              │   │  loader.sh)│  │
│  │ scan cmd ◻   │   │ Scheduler ✓  │   │            │  │
│  │ upload cmd ◻ │   │ Heartbeat ✓  │   │ Core ✓     │  │
│  │ push cmd ◻   │   │ C2 handler ✓ │   │ Enterprise │  │
│  │ diff cmd ◻   │   │ Retry ✓      │   │  (license) │  │
│  │ status cmd ◻ │   │ Systemd ✓    │   │            │  │
│  └──────┬───────┘   └──────┬───────┘   └─────┬──────┘  │
│         │                  │                  │         │
│         │  BashRunner      │  source          │  hooks  │
│         ▼                  ▼                  ▼         │
│  ┌─────────────────────────────────────────────────┐    │
│  │          Bash Scripts (50+ modules)              │    │
│  │  discover_*, enum_*, vuln_*, report_*, risk_*   │    │
│  └─────────────────────┬───────────────────────────┘    │
│                        │                                │
│                        │ upload / heartbeat / C2        │
└────────────────────────┼────────────────────────────────┘
                         │
                         ▼
              ┌─────────────────────┐
              │   bullium-site      │
              │   (Netlify Portal)  │
              │                     │
              │ /api/upload         │
              │ /api/health         │
              │ /api/agent-commands │
              │ /api/agents         │
              │ /api/webhooks       │
              │ Neon PostgreSQL     │
              │ Netlify Blobs       │
              └─────────────────────┘

✓ = implemented   ◻ = stubbed

Known Integration Friction Points

These are architectural risks that grow worse as more features are added. Address them proactively to avoid painful refactors later.

1. Config Key Sprawl (Priority: HIGH)

Problem: Config keys are defined ad-hoc across Python, daemon, plugins, and individual scripts. No central registry means typos are silently ignored, documentation drifts from implementation, and new contributors can't discover valid options.

Current state: ~40+ config keys across report_config.conf.example, daemon globals, Python config.py, and plugin manifests.

Recommended fix: Create lib/config_schema.sh, a central registry of all valid keys with types, defaults, and descriptions. Both Python and Bash config loaders validate against this schema.

# lib/config_schema.sh (example)
# Format: KEY|TYPE|DEFAULT|DESCRIPTION
_NV_CONFIG_SCHEMA=(
  "TARGETS|string||Scan target(s): CIDR, hostname, or IP"
  "SCAN_PROFILE|string|full|Scan profile name"
  "DAEMON_ENABLED|bool|false|Enable daemon scheduling for this config"
  "DAEMON_POLL_INTERVAL|int|60|Seconds between schedule checks"
  # ...
)

2. No API Versioning Between Daemon and Portal (Priority: HIGH)

Problem: The daemon assumes the portal responds with a .commands array in the heartbeat response, and the portal assumes the daemon supports all command types. Neither side advertises its version or capabilities. If the portal adds a new command type, older daemons will fail silently.

Recommended fix: Add version negotiation to the heartbeat:

• Daemon side: Include protocol_version and supported_commands[] in heartbeat payload
• Portal side: Only dispatch commands the agent supports; include min_agent_version in command payloads for safety
• Add protocol_version to the API contract

3. Shared State Race Conditions (Priority: MEDIUM)

Problem: Both daemon and Python pipeline write session_results.json. If a scan is running via daemon and a manual upload happens concurrently, session state can be corrupted.

Current mitigations: None, no file locking, no optimistic concurrency.

Recommended fix: Use flock(1) for Bash, fcntl.lockf for Python, both on a session-level lock file ($SESSION_DIR/.lock). Short-term, just document that manual operations should not overlap with daemon scans.

4. C2 Command Persistence Gap (Priority: MEDIUM): Phase B resolved (#84)

Problem: update_schedule and push_config C2 commands modified daemon state in memory only. On daemon restart or SIGHUP reload, changes reverted to what's in the config file.

Status: push_config is now persisted (issue #84). Accepted allowlist keys are written to <configs_dir>/.c2_overrides.conf and re-applied by _nvd_load_configs() as the final layer after the .conf files, so a pushed config change survives reload and restart (override wins over the config file). See Command & Control section 6. update_schedule remains in-memory only.

Remaining (phased):

• Phase A: Surface the (now narrow) limitation in the admin UI (update_schedule).
• Phase B: Done for push_config via .c2_overrides.conf.
• Phase C: Add portal-side "pending config" re-dispatched on agent reconnect, and extend persistence to update_schedule.

5. Plugin Dependency Cycle Risk (Priority: LOW)

Problem: plugin_loader.sh does topological sort on dependencies but has no cycle detection. Circular deps will infinite loop.

Recommended fix: Track visited set during DFS; if a node is revisited before completion, log error and skip the cycle.

6. Python CLI Completion (Priority: LOW, until needed)

Current state: Only recon command is wired. scan, upload, push, diff, and status are stubbed.

Recommendation: Complete these as the Python wrapper becomes the primary entry point. Priority order:

1. upload / push: needed for daemon Python migration
2. status: useful for monitoring
3. diff: useful for reporting
4. scan: lower priority (recon subsumes it)

Dependency Direction Rule

When choosing which subsystem depends on which, follow this principle:

Python CLI ──depends on──▶ Bash scripts (via BashRunner)
Bash daemon ──depends on──▶ Bash scripts (via source)
Plugin loader ──depends on──▶ Bash scripts (via hooks)
Bash scripts ──depends on──▶ NOTHING above (stay independent)

Why: Bash scripts are the stable foundation. They should never import or call the Python wrapper, daemon, or plugin loader. This keeps individual scripts testable and runnable standalone. Higher-level orchestrators (Python, daemon, plugins) compose the scripts: scripts don't know about orchestrators.

Future Architecture Decision Points

These are forks in the road that don't need to be decided yet, but should be discussed when they become relevant:

1. Python vs Bash daemon: Should netvuln_daemon.sh eventually be rewritten in Python? The Python orchestrator already has config loading and subprocess management. But the Bash daemon is battle-tested and signal handling in Python is trickier.

2. Plugin system scope: Should plugins be able to register new C2 command types? Currently, command types are hardcoded in command_handler.sh. Plugin-registered commands would need a dispatch table.

3. Portal-side agent management: As the fleet grows, should agents register capabilities (installed tools, platform, network access) so the portal can make smarter command dispatch decisions?

4. Config as code: Should scanner configs eventually live in the portal (downloadable by agents) rather than as local files? This would solve the C2 config persistence gap but adds portal dependency for cold starts.

Cross-Repository Testing Strategy

Before Merging Portal Changes

1. Run full test suite: npm test (2106 tests)
2. Deploy to dev: merge feature branch → dev
3. Run E2E smoke test from netvuln-tool against dev deploy:

   # Point smoke test at the dev deploy URL
   # Create a temp config pointing at the dev deploy
   echo 'CONSULTATION_API_URL="https://dev--bullium.netlify.app/api/upload"' > /tmp/dev-test.conf
   echo 'NV_API_KEY="nvt_pro_<hex>"' >> /tmp/dev-test.conf
   bash scripts/e2e_smoke_test.sh -c /tmp/dev-test.conf

4. If schema changed: verify functions work against dev Neon branch
5. Validate via Netlify preview deploy (visual check)

Before Merging Scanner Changes

1. Run BATS tests: bats tests/ (1088 tests)
2. Run client JS tests: npm run test:client (143 tests)
3. If upload fields changed: test upload against dev portal deploy
4. If report template changed: verify share page rendering
5. Run shellcheck on modified scripts

Integration Test Workflow

For changes touching both repos (e.g., new metadata field):

1. Portal: feature branch → add field support → merge to dev → deploy
2. Scanner: feature branch → add field generation → test against dev deploy
3. Portal: dev → main (production)
4. Scanner: merge feature branch → tag release
5. Verify: upload from updated scanner to production portal