docs: add archive upload/copy-path implementation plan
This commit is contained in:
Andrew-AAAA committed
1 parent
4b33af54ae
commit
4606c0adeb
1 file changed
+224
@@ -0,0 +1,224 @@
|
||||
# Archive Upload + Copy-Path Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development (recommended) or superpowers:executing-plans. Checkbox steps.
|
||||
|
||||
**Goal:** Let user copy archive path as Windows/Linux path, and upload videos + create date folders from the Video Archive page.
|
||||
|
||||
**Architecture:** Two new library endpoints stream files into the archive (compose mount flips `:ro` → rw); project payload gains `video_root_linux`/`video_root_windows` derived from env; one new component `ArchiveControls.jsx` holds copy buttons + folder/upload UI; docs REQ-010/REQ-163 revised (user approved).
|
||||
|
||||
**Tech Stack:** FastAPI + `library.py` path helpers, React 19, nginx proxy, Docker Compose.
|
||||
|
||||
**Spec:** `docs/requirements.md` (REQ-010, REQ-163 revised; REQ-171, REQ-172 new), `docs/design.md` API table.
|
||||
|
||||
## Global Constraints
|
||||
- Commit only files named in your task brief; never `git add -A` (the tree carries unrelated dirty files)
|
||||
- Archive paths only via `library._safe_join` — no traversal (`backend/library.py:19`)
|
||||
- Date folders: `^\d{4}-\d{2}-\d{2}$`; extensions: `video.VIDEO_EXTS = (".mp4",".mkv",".mov",".avi",".webm",".m4v")` (`backend/video.py:15`)
|
||||
- No delete/rename UI (user decision)
|
||||
- New frontend file ≤400 lines; existing `LibraryPage.jsx` gets small diff only
|
||||
- UI checklist: no emoji, `cursor:pointer`, hover 150-300ms, focus visible, contrast ≥4.5:1, `prefers-reduced-motion`, responsive
|
||||
- Page copy is Indonesian-dominant → new button labels Indonesian
|
||||
- Verification = curl + browser (repo has no test framework)
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Docs — REQ revisions + new REQs
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/requirements.md`
|
||||
- Modify: `docs/design.md` (API table ~line 402-405 area, add rows)
|
||||
- Modify: `docs/tasks.md` (add 3 TODO entries)
|
||||
|
||||
**Steps:**
|
||||
- [ ] Revise **REQ-010** → text: "The video archive lives at a local path (disk or mount). Videos may be copied in from disk **or uploaded through the browser** into an existing date folder (REQ-171)."
|
||||
- [ ] Revise **REQ-163** → "Nothing existing in the archive is moved, renamed or deleted; only user-initiated uploads and folder creation append to it."
|
||||
- [ ] Add **REQ-171** — "The Video Archive page can create a `YYYY-MM-DD` folder and upload videos into it. Uploads are streamed to disk, never overwrite an existing file, and only accept `video.VIDEO_EXTS`."
|
||||
- [ ] Add **REQ-172** — "The Video Archive page shows a button that copies the archive's host path to the clipboard, as a Linux path and as a Windows (`\\wsl.localhost\<distro>\...`) path when known."
|
||||
- [ ] Add design.md API rows: `POST /api/projects/{id}/library/dates?date=`, `POST /api/projects/{id}/library/upload?date=` (multipart `file`), project payload +`video_root_linux`,`video_root_windows`
|
||||
- [ ] Add tasks.md entries with below verification criteria, status `[TODO]`
|
||||
|
||||
### Task 2: Backend — host paths in project payload + env passthrough
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/config.py` (add function after `VIDEO_ROOT`, line 23)
|
||||
- Modify: `backend/projects.py:180` (serializer dict)
|
||||
- Modify: `docker-compose.yml` (backend `environment:`)
|
||||
- Modify: `.env` (absolute `VIDEO_ARCHIVE_HOST`, add `WSL_DISTRO_NAME`), `.env.example` (same + comment)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `config.archive_host_paths() -> {"linux": str|None, "windows": str|None}`; payload keys `video_root_linux`, `video_root_windows`
|
||||
|
||||
**Steps:**
|
||||
- [ ] Add to `backend/config.py`:
|
||||
```python
|
||||
def archive_host_paths() -> dict:
|
||||
"""Where the archive sits on the host — for the copy-path buttons (REQ-172).
|
||||
|
||||
ponytail: relative VIDEO_ARCHIVE_HOST (./data/archive) cannot be resolved
|
||||
inside the container; set it absolute in .env or the buttons stay hidden."""
|
||||
import re
|
||||
host = os.environ.get("VIDEO_ARCHIVE_HOST", "")
|
||||
if os.path.isabs(host):
|
||||
linux = host.rstrip("/")
|
||||
elif VIDEO_ROOT != "/videos": # bare-metal: VIDEO_ROOT is already the host path
|
||||
linux = VIDEO_ROOT
|
||||
else:
|
||||
return {"linux": None, "windows": None}
|
||||
if re.match(r"^[A-Za-z]:[\\/]", linux):
|
||||
return {"linux": None, "windows": linux.replace("/", "\\")}
|
||||
distro = os.environ.get("WSL_DISTRO_NAME", "")
|
||||
windows = f"\\\\wsl.localhost\\{distro}\\" + linux.lstrip("/") if distro else None
|
||||
return {"linux": linux, "windows": windows}
|
||||
```
|
||||
- [ ] In project serializer dict (`backend/projects.py:180`) add:
|
||||
```python
|
||||
"video_root_linux": config.archive_host_paths()["linux"],
|
||||
"video_root_windows": config.archive_host_paths()["windows"],
|
||||
```
|
||||
- [ ] `docker-compose.yml` backend environment add:
|
||||
```yaml
|
||||
VIDEO_ARCHIVE_HOST: ${VIDEO_ARCHIVE_HOST:-./data/archive}
|
||||
WSL_DISTRO_NAME: ${WSL_DISTRO_NAME:-}
|
||||
```
|
||||
- [ ] `.env`: `VIDEO_ARCHIVE_HOST=/home/araaraenjoyer/dbs_project/reTraining/data/archive`, add `WSL_DISTRO_NAME=Ubuntu`; mirror in `.env.example` with comment "must be absolute for the copy-path buttons (REQ-172)"
|
||||
- [ ] Verify: rebuild backend, `curl -s localhost:9010/api/projects/1` → keys present, windows value `\\wsl.localhost\Ubuntu\home\...`
|
||||
|
||||
### Task 3: Backend — mkdir + upload endpoints
|
||||
|
||||
**Files:**
|
||||
- Modify: `backend/library.py` (after `list_videos`, ~line 99)
|
||||
- Modify: `backend/api/projects.py` (after line 151)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `_safe_join`, `_effective_root`, `video.VIDEO_EXTS`
|
||||
- Produces: `library.LibraryConflict(LibraryError)`; `library.create_date(video_root, date) -> str`; `library.upload_video(video_root, date, filename, chunks) -> str` (returns `rel`); endpoints `POST /library/dates` → `{"date"}` (409 if exists), `POST /library/upload` → `{"rel"}` (409 exists, 400 bad ext/date/name)
|
||||
|
||||
**Steps:**
|
||||
- [ ] Add to `backend/library.py`:
|
||||
```python
|
||||
class LibraryConflict(LibraryError):
|
||||
pass
|
||||
|
||||
|
||||
def create_date(video_root: str, date: str) -> str:
|
||||
"""Make a YYYY-MM-DD folder; never touch an existing one (REQ-171)."""
|
||||
if not re.fullmatch(r"\d{4}-\d{2}-\d{2}", date):
|
||||
raise LibraryError("Folder name must be YYYY-MM-DD")
|
||||
folder = _safe_join(_effective_root(video_root), date)
|
||||
if os.path.exists(folder):
|
||||
raise LibraryConflict(f"Folder already exists: {date}")
|
||||
os.makedirs(folder)
|
||||
return folder
|
||||
|
||||
|
||||
def upload_video(video_root: str, date: str, filename: str, chunks) -> str:
|
||||
"""Stream `chunks` into <date>/<filename>; atomic via .part + rename."""
|
||||
name = os.path.basename(filename or "")
|
||||
if not name or name != filename or not name.lower().endswith(video.VIDEO_EXTS):
|
||||
raise LibraryError(f"Unsupported file: {filename}")
|
||||
root = _effective_root(video_root)
|
||||
if not os.path.isdir(_safe_join(root, date)):
|
||||
raise LibraryError(f"No such date in the archive: {date}")
|
||||
dest = _safe_join(root, date, name)
|
||||
if os.path.exists(dest):
|
||||
raise LibraryConflict(f"Already exists: {date}/{name}")
|
||||
part = dest + ".part"
|
||||
try:
|
||||
with open(part, "wb") as handle:
|
||||
for chunk in chunks:
|
||||
handle.write(chunk)
|
||||
os.replace(part, dest)
|
||||
except BaseException:
|
||||
if os.path.exists(part):
|
||||
os.unlink(part)
|
||||
raise
|
||||
return f"{date}/{name}"
|
||||
```
|
||||
- [ ] Add endpoints to `backend/api/projects.py` (`File`, `UploadFile` already imported line 8):
|
||||
```python
|
||||
@router.post("/{project_id}/library/dates")
|
||||
def create_library_date(project_id: int, date: str) -> dict:
|
||||
project = project_or_404(project_id)
|
||||
try:
|
||||
library.create_date(project["video_root"], date)
|
||||
except library.LibraryConflict as exc:
|
||||
raise HTTPException(409, str(exc))
|
||||
except library.LibraryError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
return {"date": date}
|
||||
|
||||
|
||||
@router.post("/{project_id}/library/upload")
|
||||
def upload_library_video(project_id: int, date: str,
|
||||
file: UploadFile = File(...)) -> dict:
|
||||
project = project_or_404(project_id)
|
||||
try:
|
||||
rel = library.upload_video(
|
||||
project["video_root"], date, file.filename or "",
|
||||
iter(lambda: file.file.read(1 << 20), b""))
|
||||
except library.LibraryConflict as exc:
|
||||
raise HTTPException(409, str(exc))
|
||||
except library.LibraryError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
return {"rel": rel}
|
||||
```
|
||||
- [ ] Verify (deferred to integration phase — needs Docker rebuild after Tasks 2-4): mkdir 200/409/400, upload 200/409/400, list shows file, `.part` cleaned, shell `rm -rf` test folder afterwards
|
||||
|
||||
### Task 4: Infra — rw mount + nginx body size
|
||||
|
||||
**Files:**
|
||||
- Modify: `docker-compose.yml` (archive volume line `...:/videos:ro` → drop `:ro`)
|
||||
- Modify: `frontend/nginx.conf:7` (`client_max_body_size 512m` → `20g`, update comment)
|
||||
|
||||
**Steps:**
|
||||
- [ ] Edit both; build deferred to integration phase
|
||||
- [ ] Verify (integration): `docker inspect retraining-backend-1 --format '{{json .Mounts}}'` shows rw (no `"RO":true`); `curl -s localhost:9010/api/health` → 200
|
||||
|
||||
### Task 5: Frontend — api helpers + ArchiveControls component + LibraryPage wiring
|
||||
|
||||
**Files:**
|
||||
- Modify: `frontend/src/api.js` (after line 65)
|
||||
- Create: `frontend/src/components/ArchiveControls.jsx`
|
||||
- Modify: `frontend/src/pages/LibraryPage.jsx` (~line 474-498 header, ~line 399 load effect)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: payload keys `video_root_linux`, `video_root_windows`; endpoints Task 3
|
||||
- Produces: `api.createLibraryDate(id, date)`, `api.uploadLibraryVideo(id, date, file, onProgress)`; component `<ArchiveControls project={project} projectId={projectId} onChanged={reloadArchive} />`
|
||||
|
||||
**Steps:**
|
||||
- [ ] `frontend/src/api.js`:
|
||||
```js
|
||||
createLibraryDate: (id, date) =>
|
||||
request(`/projects/${id}/library/dates?date=${encodeURIComponent(date)}`, { method: 'POST' }),
|
||||
uploadLibraryVideo: (id, date, file, onProgress) =>
|
||||
new Promise((resolve, reject) => {
|
||||
const form = new FormData()
|
||||
form.append('file', file)
|
||||
const xhr = new XMLHttpRequest()
|
||||
xhr.open('POST', `/api/projects/${id}/library/upload?date=${encodeURIComponent(date)}`)
|
||||
xhr.upload.onprogress = (e) => { if (e.lengthComputable) onProgress(e.loaded / e.total) }
|
||||
xhr.onload = () => {
|
||||
let payload = {}
|
||||
try { payload = JSON.parse(xhr.responseText || '{}') } catch {}
|
||||
if (xhr.status >= 200 && xhr.status < 300) resolve(payload)
|
||||
else reject(new Error(payload.detail || `${xhr.status} ${xhr.statusText}`))
|
||||
}
|
||||
xhr.onerror = () => reject(new Error('network error'))
|
||||
xhr.send(form)
|
||||
}),
|
||||
```
|
||||
- [ ] Create `frontend/src/components/ArchiveControls.jsx` (~170 lines) with:
|
||||
- `CopyPath`: two `btn btn-ghost` buttons rendered only when `project.video_root_linux` / `_windows` truthy; `navigator.clipboard.writeText` → label swap "Tersalin" 1.5s (`aria-live="polite"`); `title=` shows full path
|
||||
- `New folder`: `<input type="date">` + button → `api.createLibraryDate` → `onChanged()`; conflict (409) → error text
|
||||
- `Upload`: hidden `<input type="file" multiple accept=".mp4,.mkv,.mov,.avi,.webm,.m4v">`; button "Unggah video"; sequential `api.uploadLibraryVideo` per file with thin progress bar (width % via inline style; CSS transition 200ms, disabled under `prefers-reduced-motion`); per-file error rows; `onChanged()` after each success
|
||||
- All: `cursor:pointer`, hover opacity/transition 200ms, visible `:focus-visible` outline (match existing `btn` styles), no emoji, 4.5:1 text
|
||||
- [ ] `LibraryPage.jsx` wiring:
|
||||
- add `const reloadArchive = useCallback(() => { api.archiveCycles(projectId).then(l => { setDates(l.cycles); if (!l.cycles.some(c => c.cycle === selected)) setSelected(l.cycles[0]?.cycle ?? null) }).catch(() => {}) }, [projectId, selected])`
|
||||
- render `<ArchiveControls project={project} projectId={projectId} onChanged={reloadArchive} />` inside page-head right column above the truck-scan button (line ~484), stack vertically with `gap: 8`
|
||||
- [ ] Verify (integration phase): build frontend, browser checks + AGENTS §8 checklist
|
||||
|
||||
### Task 6: Docs close-out
|
||||
|
||||
**Steps:**
|
||||
- [ ] Flip new `docs/tasks.md` entries to `[DONE]` with verification evidence
|
||||
- [ ] Re-read `docs/design.md` API table + disk layout — matches shipped behavior (no doc lies)
|
||||
Reference in new issue
Block a user