argus/backend
Watch
2
0
Fork
You've already forked backend
 0
Code Issues Pull requests Projects Releases 1 Packages 1 Wiki Activity Actions 1

Source refs, JSON uploads and de-duplication #32

Merged
Gilles merged 9 commits from feat/28-sources-json-dedup into master 2026-10-28 22:34:03 +00:00
Conversation 1 Commits 9 Files changed 13 +395 -11
Gilles commented 2026-10-28 15:26:04 +00:00
Owner

Closes #28

Part of the server monitoring plan (phase 1.3). Stacked on #31 (base feat/27-authentik-group-access): merge #30 and #31 first.

Changes

  • Report.RefKind.SOURCE (source): ref_value names a machine or tool (garion, kuma…). Migrations only alter choices (reports and events) and add Report.last_seen_at.
  • Upload endpoint: a JSON body {"report_type": "custom", …, "report": {…}} is accepted instead of a file (custom reports only; exactly one of file/report). commit_sha is optional; without it the raw file goes to <owner>/<repo>/<ref_kind>-<ref_value>/<timestamp>/…, so uploads never overwrite each other.
  • Format maison: optional metric key (JSON/YAML/XML), report-level expect_every parsed by the new apps.reports.durations and stored in the summary in seconds (dropped when unparseable). Unknown to older clients, nothing breaks.
  • De-duplication (process_report): on a source ref, a report without detail whose status and summary equal the latest processed report of the same stream is deleted (row + raw file); that report's last_seen_at moves forward instead. No event, no aggregate recompute. Branch/commit/PR reports are never de-duplicated.
  • prune_history also bounds source refs; event subjects read "source garion".
  • argus/upload-action needs no change (it always sends a commit).
  • README: how to push from a script; feature catalog entries; tests in apps/reports/test_sources.py.

Checks

pytest (209 passed), pylint 10/10, autopep8, bandit clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_014BSmBtGWeFBD9z4mYm9SjA

Closes #28 Part of the server monitoring plan (phase 1.3). **Stacked on #31** (base `feat/27-authentik-group-access`): merge #30 and #31 first. ## Changes - `Report.RefKind.SOURCE` (`source`): `ref_value` names a machine or tool (`garion`, `kuma`…). Migrations only alter `choices` (reports and events) and add `Report.last_seen_at`. - Upload endpoint: a JSON body `{"report_type": "custom", …, "report": {…}}` is accepted instead of a file (custom reports only; exactly one of `file`/`report`). `commit_sha` is optional; without it the raw file goes to `<owner>/<repo>/<ref_kind>-<ref_value>/<timestamp>/…`, so uploads never overwrite each other. - Format maison: optional metric `key` (JSON/YAML/XML), report-level `expect_every` parsed by the new `apps.reports.durations` and stored in the summary in seconds (dropped when unparseable). Unknown to older clients, nothing breaks. - De-duplication (`process_report`): on a `source` ref, a report without detail whose status and summary equal the latest processed report of the same stream is deleted (row + raw file); that report's `last_seen_at` moves forward instead. No event, no aggregate recompute. Branch/commit/PR reports are never de-duplicated. - `prune_history` also bounds `source` refs; event subjects read "source garion". - `argus/upload-action` needs no change (it always sends a commit). - README: how to push from a script; feature catalog entries; tests in `apps/reports/test_sources.py`. ## Checks pytest (209 passed), pylint 10/10, autopep8, bandit clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_014BSmBtGWeFBD9z4mYm9SjA
feature: source refs, JSON uploads and de-duplication
Some checks failed
ci / test (pull_request) Failing after 3m3s
Details
ci / e2e (pull_request) Has been skipped
Details
ci / docker (pull_request) Has been skipped
Details
ci / deploy (pull_request) Has been skipped
Details
5f84d1a5b5 
- ref_kind=source (a machine or tool name), allowed on any project.
- The upload endpoint also takes a JSON body with the "format maison"
  report itself, no file; commit_sha becomes optional (a commit-less
  upload gets its own storage directory).
- Format maison: optional stable metric `key`, and `expect_every`
  (90s/15m/1h/7d...) kept in the summary in seconds.
- On a source ref, a report identical to the latest one of its stream
  is dropped and only moves that one's new last_seen_at forward.
- History pruning also bounds source refs.

Closes #28

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BSmBtGWeFBD9z4mYm9SjA
Gilles commented 2026-10-28 15:29:04 +00:00
Author
Owner

📊 Rapports Argus pour cette PR

<!-- argus-link-bot:argus/backend --> 📊 [Rapports Argus pour cette PR](https://reports.stationmyr.net/argus/backend/pr/32)
Merge branch 'feat/27-authentik-group-access' into feat/28-sources-json-dedup
All checks were successful
ci / test (pull_request) Successful in 3m11s
Details
ci / e2e (pull_request) Has been skipped
Details
ci / docker (pull_request) Has been skipped
Details
ci / deploy (pull_request) Has been skipped
Details
790efa831d
Merge branch 'feat/27-authentik-group-access' into feat/28-sources-json-dedup
All checks were successful
ci / test (pull_request) Successful in 3m10s
Details
ci / e2e (pull_request) Has been skipped
Details
ci / docker (pull_request) Has been skipped
Details
ci / deploy (pull_request) Has been skipped
Details
31e47f6e55
Gilles changed target branch from feat/27-authentik-group-access to master 2026-10-28 19:03:51 +00:00
Merge remote-tracking branch 'origin/master' into feat/28-sources-json-dedup
Some checks failed
ci / test (pull_request) Has been cancelled
Details
ci / e2e (pull_request) Has been cancelled
Details
ci / docker (pull_request) Has been cancelled
Details
ci / deploy (pull_request) Has been cancelled
Details
ccfdb6546a 
# Conflicts:
#	README.md
fix: drop the access-groups block the master merge duplicated in the feature catalog
All checks were successful
ci / test (pull_request) Successful in 3m10s
Details
ci / e2e (pull_request) Has been skipped
Details
ci / docker (pull_request) Has been skipped
Details
ci / deploy (pull_request) Has been skipped
Details
6b50d3da9a 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BSmBtGWeFBD9z4mYm9SjA
Gilles merged commit 6a9da6891c into master 2026-10-28 22:34:03 +00:00
Gilles deleted branch feat/28-sources-json-dedup 2026-10-28 22:34:03 +00:00
Gilles referenced this pull request from a commit 2026-10-29 06:56:29 +00:00
Show the sources of a project (#22)
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
breaking
Breaking change: listed first in the release notes

  
bug
Bug fix (release notes: Bug fixes)

  
ci
CI and maintenance (release notes: CI & maintenance)

  
dependencies
Dependency update (release notes: Dependencies, folded)

  
documentation
Documentation (release notes: Documentation)

  
feature
New feature (release notes: Features)

  
security
Security fix (release notes: Security)

  
skip-changelog
Left out of the release notes

No labels
breaking
bug
ci
dependencies
documentation
feature
security
skip-changelog
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
argus/backend!32
No description provided.