feat: use EMAC zone API for paginated genre scraping

Previously only ~17 IDs were extracted from initial HTML per genre page
(before the "voir plus" button). The EMAC API at api.arte.tv exposes all
concerts with full pagination (e.g. 131 Metal, 187 Pop Rock).

Also reuses metadata from EMAC response (title, subtitle, thumbnail,
expiry) — skipping redundant player API calls for genre concerts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Dockerfile

@@ -2,13 +2,19 @@ FROM python:3.12-slim
 
 WORKDIR /app
 
-RUN apt-get update -qq && apt-get install -y -qq ffmpeg && rm -rf /var/lib/apt/lists/*
+RUN apt-get update -qq \
+    && apt-get install -y -qq ffmpeg gosu \
+    && rm -rf /var/lib/apt/lists/* \
+    && groupadd -r abc \
+    && useradd -r -g abc abc
 
 COPY requirements.txt .
 RUN pip install --no-cache-dir -r requirements.txt
 
 COPY . .
+RUN chmod +x /app/entrypoint.sh
 
 EXPOSE 8080
 
+ENTRYPOINT ["/app/entrypoint.sh"]
 CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

README.md

@@ -1,3 +1,128 @@
 # arte-dl
 
-Web GUI pour parcourir et télécharger les concerts Arte Concert
+Web GUI pour parcourir et télécharger les concerts Arte Concert.
+
+- Navigation par catégorie (Metal, Jazz, Classique, Opéra…)
+- Téléchargement en WEB-DL avec nommage scène (`-ReMoRa`)
+- Enrichissement TMDB (poster, backdrop)
+- Auto-téléchargement par catégorie (souscription persistante)
+
+---
+
+## Déploiement
+
+### Prérequis
+
+- Docker + Docker Compose
+- Une clé API TMDB (gratuite sur [themoviedb.org](https://www.themoviedb.org/settings/api))
+
+### Docker Compose (générique)
+
+```yaml
+services:
+  arte-dl:
+    image: forge.dilain.com/laurent/arte-dl:latest
+    container_name: arte-dl
+    restart: unless-stopped
+    ports:
+      - "8085:8080"
+    volumes:
+      - /chemin/vers/media/Arte:/data/Arte      # fichiers téléchargés
+      - /chemin/vers/appdata/arte-dl:/app/data  # base SQLite
+    environment:
+      - TZ=Europe/Paris
+      - TMDB_API_KEY=your_tmdb_api_key
+```
+
+```bash
+docker compose up -d
+```
+
+L'interface est accessible sur `http://localhost:8085`.
+
+---
+
+### Déploiement sur Unraid
+
+#### Via le Community Applications (template manuel)
+
+1. Dans Unraid, aller dans **Docker** → **Add Container**
+2. Remplir les champs :
+
+| Champ | Valeur |
+|-------|--------|
+| Name | `arte-dl` |
+| Repository | `forge.dilain.com/laurent/arte-dl:latest` |
+| Network Type | `bridge` |
+| Port (Host→Container) | `8085 → 8080` |
+
+3. Ajouter les **Path mappings** :
+
+| Name | Container Path | Host Path | Access |
+|------|---------------|-----------|--------|
+| Media | `/data/Arte` | `/mnt/user/data/Arte` | Read/Write |
+| Config | `/app/data` | `/mnt/user/appdata/arte-dl` | Read/Write |
+
+4. Ajouter les **variables d'environnement** :
+
+| Name | Value |
+|------|-------|
+| `TZ` | `Europe/Paris` |
+| `TMDB_API_KEY` | `<votre clé TMDB>` |
+| `AUTO_DL_INTERVAL` | `3600` *(optionnel, intervalle auto-DL en secondes)* |
+
+5. Cliquer **Apply**
+
+#### Via docker-compose sur Unraid
+
+Copier le fichier `docker-compose.yml` dans `/mnt/user/appdata/arte-dl/` et lancer :
+
+```bash
+cd /mnt/user/appdata/arte-dl
+TMDB_API_KEY=xxx docker compose up -d
+```
+
+#### Volumes Unraid
+
+```
+/mnt/user/data/Arte          → /data/Arte    (concerts téléchargés, par sous-dossier de catégorie)
+/mnt/user/appdata/arte-dl    → /app/data     (arte_dl.db — souscriptions, historique, cache TMDB)
+```
+
+> **Note :** Le dossier `/data/Arte` est créé automatiquement au premier téléchargement.  
+> Les sous-dossiers de catégorie (`/data/Arte/Metal/`, `/data/Arte/Jazz/`…) sont créés selon la catégorie active lors du téléchargement.
+
+---
+
+## Variables d'environnement
+
+| Variable | Défaut | Description |
+|----------|--------|-------------|
+| `TMDB_API_KEY` | — | Clé API TMDB (obligatoire pour posters/backdrops) |
+| `TZ` | `UTC` | Fuseau horaire |
+| `AUTO_DL_INTERVAL` | `3600` | Intervalle (secondes) entre deux checks auto-DL |
+| `PUID` | `0` | UID Unix du propriétaire des fichiers (Unraid : `99`) |
+| `PGID` | `0` | GID Unix du propriétaire des fichiers (Unraid : `100`) |
+
+---
+
+## Auto-téléchargement
+
+Chaque pill de catégorie dispose d'un bouton **⬇** :
+- Clic sur **⬇** → souscription activée (icône dorée)
+- Les nouveaux concerts de cette catégorie sont téléchargés automatiquement toutes les `AUTO_DL_INTERVAL` secondes
+- Les souscriptions sont persistées en base SQLite (survivent aux redémarrages)
+- Déclenchement immédiat possible via `POST /api/auto-dl/check`
+
+---
+
+## Build local
+
+```bash
+docker build -t arte-dl .
+docker run -p 8085:8080 \
+  -v /chemin/Arte:/data/Arte \
+  -v /chemin/data:/app/data \
+  -e TMDB_API_KEY=xxx \
+  arte-dl
+```

arte_api.py

@@ -1,4 +1,5 @@
 import re
+import sqlite3
 import time
 import logging
 import asyncio
@@ -11,7 +12,63 @@ import tmdb as _tmdb
 logger = logging.getLogger(__name__)
 
 CACHE_TTL = 6 * 3600
+DB_PATH = "data/arte_dl.db"
 _cache: dict = {"data": [], "ts": 0}
+_fetch_lock: asyncio.Lock | None = None
+_refresh_task: asyncio.Task | None = None
+
+
+def _get_fetch_lock() -> asyncio.Lock:
+    global _fetch_lock
+    if _fetch_lock is None:
+        _fetch_lock = asyncio.Lock()
+    return _fetch_lock
+
+
+import os as _os
+_os.makedirs("data", exist_ok=True)
+
+
+def _db():
+    conn = sqlite3.connect(DB_PATH)
+    conn.row_factory = sqlite3.Row
+    return conn
+
+
+def _init_concerts_cache_table():
+    with _db() as conn:
+        conn.execute("""
+            CREATE TABLE IF NOT EXISTS concerts_cache (
+                id      INTEGER PRIMARY KEY CHECK (id = 1),
+                data    TEXT NOT NULL,
+                ts      REAL NOT NULL
+            )
+        """)
+
+
+_init_concerts_cache_table()
+
+
+def _load_db_cache() -> tuple[list, float]:
+    try:
+        with _db() as conn:
+            row = conn.execute("SELECT data, ts FROM concerts_cache WHERE id=1").fetchone()
+        if row:
+            return json.loads(row["data"]), row["ts"]
+    except Exception:
+        pass
+    return [], 0.0
+
+
+def _save_db_cache(data: list, ts: float):
+    try:
+        with _db() as conn:
+            conn.execute(
+                "INSERT OR REPLACE INTO concerts_cache (id, data, ts) VALUES (1, ?, ?)",
+                (json.dumps(data), ts),
+            )
+    except Exception as e:
+        logger.warning("Failed to save concerts cache: %s", e)
 
 PLAYER_API = "https://api.arte.tv/api/player/v2/config/fr/{pid}"
 SEARCH_URL = "https://www.arte.tv/fr/search/?q={q}"
@@ -53,12 +110,54 @@ def _fetch_url(url: str, headers: dict | None = None) -> str:
         return r.read().decode("utf-8", errors="replace")
 
 
-def _prog_ids_from_page(url: str) -> set[str]:
+EMAC_ZONE_API = "https://api.arte.tv/api/emac/v4/fr/web/zones/{zone_id}/content?page={page}"
+_EMAC_HEADERS = {"User-Agent": _HEADERS["User-Agent"], "Origin": "https://www.arte.tv"}
+_ZONE_RE = re.compile(r"/api/emac/v4/fr/web/zones/([a-f0-9-]+)/content\?page=1")
+
+
+def _zone_ids_from_page(url: str) -> list[str]:
     try:
-        return set(_PROG_RE.findall(_fetch_url(url)))
+        html = _fetch_url(url)
+        return _ZONE_RE.findall(html)
     except Exception as ex:
         logger.warning("Failed to fetch %s: %s", url, ex)
-        return set()
+        return []
+
+
+def _fetch_zone_concerts(zone_id: str, category: str) -> list[dict]:
+    """Fetch all concerts from a single EMAC zone (paginated)."""
+    concerts = []
+    page = 1
+    while True:
+        url = EMAC_ZONE_API.format(zone_id=zone_id, page=page)
+        try:
+            raw = _fetch_url(url, headers=_EMAC_HEADERS)
+            data = json.loads(raw)
+        except Exception as ex:
+            logger.warning("EMAC zone %s page %d failed: %s", zone_id, page, ex)
+            break
+        for item in data.get("data", []):
+            pid = item.get("programId") or ""
+            if not pid:
+                continue
+            img = item.get("mainImage") or {}
+            avail = item.get("availability") or {}
+            concerts.append({
+                "id": pid,
+                "title": item.get("title") or "",
+                "subtitle": item.get("subtitle") or "",
+                "url": item.get("url") or f"https://www.arte.tv/fr/videos/{pid}/",
+                "thumbnail": img.get("url") or "",
+                "duration": item.get("duration"),
+                "description": item.get("shortDescription") or item.get("teaserText") or "",
+                "expiry": avail.get("end") or "",
+            })
+        pagination = data.get("pagination", {})
+        if page >= pagination.get("pages", 1):
+            break
+        page += 1
+    logger.info("  %s (zone %s…) → %d concerts", category, zone_id[:8], len(concerts))
+    return concerts
 
 
 def _metadata_for_pid(pid: str) -> dict | None:
@@ -93,21 +192,41 @@ def _metadata_for_pid(pid: str) -> dict | None:
 
 
 def _fetch_all_sync() -> list[dict]:
+    by_id: dict[str, dict] = {}
     id_cats: dict[str, list[str]] = {}
+
+    # Genre pages: use EMAC zone API (paginated) — gets ALL concerts, not just initial HTML
     for name, url in GENRE_PAGES:
-        ids = _prog_ids_from_page(url)
+        zone_ids = _zone_ids_from_page(url)
-        logger.info("  %s → %d IDs", name, len(ids))
+        if not zone_ids:
-        for pid in ids:
+            logger.warning("No zone IDs found for %s (%s)", name, url)
+            continue
+        # All zones on a page are identical copies; use the first one
+        concerts = _fetch_zone_concerts(zone_ids[0], name)
+        for c in concerts:
+            pid = c["id"]
             id_cats.setdefault(pid, []).append(name)
+            if pid not in by_id:
+                by_id[pid] = c
 
-    all_ids: set[str] = set(id_cats)
+    # Extra pages: fall back to regex (no EMAC zones)
     for url in EXTRA_PAGES:
-        ids = _prog_ids_from_page(url)
+        try:
-        logger.info("  %s → %d IDs", url.split("/fr/")[1], len(ids))
+            extra_ids = set(_PROG_RE.findall(_fetch_url(url)))
-        all_ids |= ids
+        except Exception as ex:
-    logger.info("Total unique programme IDs: %d", len(all_ids))
+            logger.warning("Failed to fetch %s: %s", url, ex)
+            extra_ids = set()
+        new_ids = extra_ids - set(by_id)
+        logger.info("  %s → %d new IDs", url.split("/fr/")[1], len(new_ids))
+        if new_ids:
+            with ThreadPoolExecutor(max_workers=10) as pool:
+                results = list(pool.map(_metadata_for_pid, sorted(new_ids)))
+            for meta in results:
+                if meta and meta.get("title"):
+                    by_id[meta["id"]] = meta
 
-    concerts = _resolve_ids(all_ids)
+    logger.info("Total unique programme IDs: %d", len(by_id))
+    concerts = list(by_id.values())
     for c in concerts:
         c["categories"] = id_cats.get(c["id"], [])
 
@@ -144,15 +263,54 @@ def _search_sync(query: str) -> set[str]:
 
 # ── public API ────────────────────────────────────────────────────────────────
 
+async def _ensure_cache() -> list[dict]:
+    return await get_all_concerts()
+
+
+async def get_concerts_by_category(category: str) -> list[dict]:
+    data = await _ensure_cache()
+    return [c for c in data if category in (c.get("categories") or [])]
+
+
+async def _do_refresh():
+    """Full scrape under lock; updates in-memory + DB cache."""
+    global _refresh_task
+    async with _get_fetch_lock():
+        if _cache["data"] and time.time() - _cache["ts"] < CACHE_TTL:
+            return
+        loop = asyncio.get_event_loop()
+        data = await loop.run_in_executor(None, _fetch_all_sync)
+        if data:
+            ts = time.time()
+            _cache["data"] = data
+            _cache["ts"] = ts
+            _save_db_cache(data, ts)
+            logger.info("Cache refreshed: %d concerts", len(data))
+    _refresh_task = None
+
+
 async def get_all_concerts() -> list[dict]:
+    global _refresh_task
     now = time.time()
+
+    # In-memory cache hit
     if _cache["data"] and now - _cache["ts"] < CACHE_TTL:
         return _cache["data"]
-    loop = asyncio.get_event_loop()
+
-    data = await loop.run_in_executor(None, _fetch_all_sync)
+    # Try DB cache — return immediately even if stale, refresh in background
-    if data:
+    db_data, db_ts = _load_db_cache()
-        _cache["data"] = data
+    if db_data:
-        _cache["ts"] = now
+        _cache["data"] = db_data
+        _cache["ts"] = db_ts
+        logger.info("Concerts loaded from DB cache (%d concerts)", len(db_data))
+        if now - db_ts >= CACHE_TTL:
+            # Stale — serve now, refresh silently in background
+            if _refresh_task is None or _refresh_task.done():
+                _refresh_task = asyncio.create_task(_do_refresh())
+        return _cache["data"]
+
+    # No cache at all — must scrape synchronously (first run or cleared DB)
+    await _do_refresh()
     return _cache["data"]
 
 
@@ -197,7 +355,41 @@ async def fetch_concerts(page: int = 1, search: str = "", page_size: int = 24, c
     }
 
 
+def get_versions(pid: str) -> list[dict]:
+    """Fetch available stream versions from Arte Player API for a programme ID."""
+    try:
+        raw = _fetch_url(
+            PLAYER_API.format(pid=pid),
+            headers={"User-Agent": _HEADERS["User-Agent"], "Accept": "application/json"},
+        )
+        data = json.loads(raw)
+        streams = data["data"]["attributes"].get("streams") or []
+        return streams[0].get("versions") or [] if streams else []
+    except Exception as ex:
+        logger.debug("Failed to get versions for %s: %s", pid, ex)
+        return []
+
+
+def select_lang_tag(versions: list[dict]) -> str:
+    """
+    Determine UNFR language tag from stream versions.
+    FR audio → FRENCH, non-FR + FR subs → VOSTFR, otherwise → VO.
+    """
+    if not versions:
+        return "VO"
+    if any(v.get("audioLanguage") == "fr" for v in versions):
+        return "FRENCH"
+    if any(v.get("subtitleLanguage") == "fr" for v in versions):
+        return "VOSTFR"
+    return "VO"
+
+
 async def invalidate_cache() -> int:
     _cache["ts"] = 0
+    try:
+        with _db() as conn:
+            conn.execute("DELETE FROM concerts_cache")
+    except Exception:
+        pass
     data = await get_all_concerts()
     return len(data)

downloader.py

@@ -1,3 +1,4 @@
+import asyncio
 import re
 import sqlite3
 import threading
@@ -7,10 +8,14 @@ from datetime import datetime
 from pathlib import Path
 
 import yt_dlp
-from fastapi import BackgroundTasks
+
+from arte_api import get_versions, select_lang_tag
 
 OUTPUT_DIR = "/data/Arte"
-DB_PATH = "arte_dl.db"
+_PID_RE = re.compile(r"\b(\d{6}-\d{3}-[A-Z])\b")
+DB_PATH = "data/arte_dl.db"
+
+Path("data").mkdir(exist_ok=True)
 
 
 def _db():
@@ -37,12 +42,10 @@ def _slugify(s: str) -> str:
     return s.strip(".")
 
 
-def build_release_name(title: str, subtitle: str, year: int | None, info: dict) -> str:
+def build_release_name(title: str, subtitle: str, year: int | None, info: dict, lang_tag: str = "VO") -> str:
+    """Build a proper UNFR/scene release name.
+    Format: Title.Event.Year.LANG.Resolution.WEB-DL.x264.AAC-ReMoRa.mkv
     """
-    Build a proper UNFR/scene release name.
-    Format: Title.Event.Year.FRENCH.Resolution.WEBRip.x264.AAC-ReMoRa.mp4
-    """
-    # Strip year from both title and subtitle to avoid duplication
     t = re.sub(r"\b" + str(year) + r"\b", "", title).strip() if year else title
     name = _slugify(t)
 
@@ -55,7 +58,6 @@ def build_release_name(title: str, subtitle: str, year: int | None, info: dict)
 
     year_str = str(year) if year else ""
 
-    # Resolution from yt-dlp info
     height = info.get("height") or 0
     if height >= 2160:
         res = "2160p"
@@ -66,7 +68,6 @@ def build_release_name(title: str, subtitle: str, year: int | None, info: dict)
     else:
         res = f"{height}p" if height else "1080p"
 
-    # Video codec (avc1 = H.264, hev1/hvc1/hevc = H.265)
     vcodec = (info.get("vcodec") or "").lower()
     if "hevc" in vcodec or "h265" in vcodec or "hev1" in vcodec or "hvc1" in vcodec:
         vc = "HEVC"
@@ -75,15 +76,16 @@ def build_release_name(title: str, subtitle: str, year: int | None, info: dict)
     else:
         vc = "x264"
 
-    parts = [name, year_str, res, "WEBRip", vc, "AAC"]
+    parts = [name, year_str, lang_tag, res, "WEB-DL", vc, "AAC"]
     base = ".".join(p for p in parts if p)
-    return f"{base}-ReMoRa.mp4"
+    return f"{base}-ReMoRa.mkv"
 
 
 class DownloadManager:
     def __init__(self):
         self._active: dict[str, dict] = {}
         self._lock = threading.Lock()
+        self._queue: asyncio.Queue = asyncio.Queue()
         self._init_db()
 
     def _init_db(self):
@@ -93,6 +95,9 @@ class DownloadManager:
                     id       TEXT PRIMARY KEY,
                     url      TEXT NOT NULL,
                     title    TEXT NOT NULL,
+                    subtitle TEXT NOT NULL DEFAULT '',
+                    year     INTEGER,
+                    category TEXT NOT NULL DEFAULT '',
                     filename TEXT,
                     state    TEXT NOT NULL DEFAULT 'queued',
                     progress REAL DEFAULT 0,
@@ -103,23 +108,95 @@ class DownloadManager:
                     error    TEXT
                 )
             """)
+            for col, definition in [
+                ("subtitle", "TEXT NOT NULL DEFAULT ''"),
+                ("year",     "INTEGER"),
+                ("category", "TEXT NOT NULL DEFAULT ''"),
+            ]:
+                try:
+                    conn.execute(f"ALTER TABLE downloads ADD COLUMN {col} {definition}")
+                except Exception:
+                    pass
+            conn.execute("""
+                CREATE TABLE IF NOT EXISTS auto_dl_categories (
+                    category TEXT PRIMARY KEY,
+                    added_at TEXT NOT NULL
+                )
+            """)
 
     # ------------------------------------------------------------------ public
 
-    def enqueue(self, url: str, title: str, subtitle: str, year: int | None,
+    def get_watched_categories(self) -> list[str]:
-                bg: BackgroundTasks) -> str:
+        with _db() as conn:
+            rows = conn.execute(
+                "SELECT category FROM auto_dl_categories ORDER BY added_at"
+            ).fetchall()
+        return [r["category"] for r in rows]
+
+    def watch_category(self, category: str):
+        with _db() as conn:
+            conn.execute(
+                "INSERT OR IGNORE INTO auto_dl_categories (category, added_at) VALUES (?,?)",
+                (category, datetime.now().isoformat()),
+            )
+
+    def unwatch_category(self, category: str):
+        with _db() as conn:
+            conn.execute("DELETE FROM auto_dl_categories WHERE category=?", (category,))
+
+    def already_enqueued(self, url: str) -> bool:
+        with _db() as conn:
+            row = conn.execute(
+                "SELECT id FROM downloads WHERE url=? AND state != 'error' LIMIT 1", (url,)
+            ).fetchone()
+        return row is not None
+
+    def already_downloaded(self, url: str) -> bool:
+        with _db() as conn:
+            row = conn.execute(
+                "SELECT id FROM downloads WHERE url=? AND state='done' LIMIT 1", (url,)
+            ).fetchone()
+        return row is not None
+
+    async def enqueue(self, url: str, title: str, subtitle: str,
+                      year: int | None, category: str) -> str:
         dl_id = str(uuid.uuid4())
         now = datetime.now().isoformat()
         with _db() as conn:
             conn.execute(
-                "INSERT INTO downloads (id, url, title, state, started_at) VALUES (?,?,?,'queued',?)",
+                "INSERT INTO downloads (id, url, title, subtitle, year, category, state, started_at) VALUES (?,?,?,?,?,?,'queued',?)",
-                (dl_id, url, title, now),
+                (dl_id, url, title, subtitle, year, category, now),
             )
         with self._lock:
             self._active[dl_id] = {"state": "queued", "progress": 0, "title": title}
-        bg.add_task(self._run, dl_id, url, title, subtitle, year)
+        await self._queue.put((dl_id, url, title, subtitle, year, category))
         return dl_id
 
+    async def resume_pending(self):
+        """Re-queue downloads interrupted by a container restart."""
+        with _db() as conn:
+            rows = conn.execute(
+                "SELECT id, url, title, subtitle, year, category FROM downloads"
+                " WHERE state IN ('queued', 'downloading') ORDER BY started_at"
+            ).fetchall()
+            conn.execute(
+                "UPDATE downloads SET state='queued', progress=0, speed='', eta=NULL"
+                " WHERE state='downloading'"
+            )
+        for r in rows:
+            with self._lock:
+                self._active[r["id"]] = {"state": "queued", "progress": 0, "title": r["title"]}
+            await self._queue.put((r["id"], r["url"], r["title"], r["subtitle"] or "", r["year"], r["category"] or ""))
+        if rows:
+            logger.info("Resumed %d pending download(s)", len(rows))
+
+    async def start_worker(self):
+        loop = asyncio.get_running_loop()
+        while True:
+            job = await self._queue.get()
+            dl_id, url, title, subtitle, year, category = job
+            await loop.run_in_executor(None, self._run, dl_id, url, title, subtitle, year, category)
+
     def status(self, dl_id: str) -> dict:
         with self._lock:
             return dict(self._active.get(dl_id, {"state": "unknown"}))
@@ -131,25 +208,30 @@ class DownloadManager:
             ).fetchall()
         return [dict(r) for r in rows]
 
-    def already_downloaded(self, url: str) -> bool:
-        with _db() as conn:
-            row = conn.execute(
-                "SELECT id FROM downloads WHERE url=? AND state='done' LIMIT 1", (url,)
-            ).fetchone()
-        return row is not None
-
     # ----------------------------------------------------------------- private
 
     def _set(self, dl_id: str, **kw):
         with self._lock:
             self._active.setdefault(dl_id, {}).update(kw)
 
-    def _run(self, dl_id: str, url: str, title: str, subtitle: str, year: int | None):
+    def _run(self, dl_id: str, url: str, title: str, subtitle: str, year: int | None, category: str = ""):
-        Path(OUTPUT_DIR).mkdir(parents=True, exist_ok=True)
+        out_dir = f"{OUTPUT_DIR}/{category}" if category else OUTPUT_DIR
+        Path(out_dir).mkdir(parents=True, exist_ok=True)
         self._set(dl_id, state="downloading")
         with _db() as conn:
             conn.execute("UPDATE downloads SET state='downloading' WHERE id=?", (dl_id,))
 
+        # Determine language tag from Arte Player API before downloading
+        pid_m = _PID_RE.search(url)
+        lang_tag = "VO"
+        if pid_m:
+            versions = get_versions(pid_m.group(1))
+            lang_tag = select_lang_tag(versions)
+
+        # MKV internal title: "Artist - Concert Title (year)"
+        name_part = f"{title} - {subtitle}" if subtitle else title
+        mkv_title = f"{name_part} ({year})" if year else name_part
+
         # For HLS, yt-dlp downloads video then audio separately.
         # After the first stream finishes, stay in "processing" to avoid
         # resetting progress to 0% when the audio stream starts.
@@ -171,22 +253,42 @@ class DownloadManager:
                 finished_once[0] = True
                 self._set(dl_id, state="processing", progress=100)
 
+        title_meta = ["-metadata", f"title={mkv_title}"]
+
         ydl_opts = {
-            "outtmpl": f"{OUTPUT_DIR}/%(title)s.%(ext)s",
+            "outtmpl": f"{out_dir}/%(title)s.%(ext)s",
             "format": "bestvideo[vcodec^=avc1]+bestaudio/bestvideo+bestaudio/best",
-            "merge_output_format": "mp4",
+            "merge_output_format": "mkv",
+            # "merger+ffmpeg_o" sets title during video+audio merge
+            "postprocessor_args": {"merger+ffmpeg_o": title_meta},
             "progress_hooks": [hook],
             "quiet": True,
             "no_warnings": True,
         }
+        if lang_tag == "VOSTFR":
+            ydl_opts.update({
+                "writesubtitles": True,
+                "subtitleslangs": ["fr"],
+                # embedsubtitles:True is CLI-only — must register the postprocessor explicitly
+                "postprocessors": [
+                    {"key": "FFmpegEmbedSubtitle", "already_have_subtitle": False},
+                ],
+            })
+            # "embedsubtitle+ffmpeg_o" sets title + default disposition during subtitle embed
+            ydl_opts["postprocessor_args"]["embedsubtitle+ffmpeg_o"] = (
+                title_meta + ["-disposition:s:0", "default"]
+            )
 
         try:
             with yt_dlp.YoutubeDL(ydl_opts) as ydl:
                 info = ydl.extract_info(url, download=True)
                 orig_path = Path(ydl.prepare_filename(info))
 
-            # Rename to proper release name
+            # yt-dlp renames to .mkv after merge; prepare_filename may return .mp4
-            release_name = build_release_name(title, subtitle, year, info)
+            if not orig_path.exists():
+                orig_path = orig_path.with_suffix(".mkv")
+
+            release_name = build_release_name(title, subtitle, year, info, lang_tag)
             dest_path = orig_path.parent / release_name
             if orig_path.exists() and orig_path != dest_path:
                 if dest_path.exists():

entrypoint.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+set -e
+
+PUID=${PUID:-0}
+PGID=${PGID:-0}
+
+if [ "$PUID" != "0" ] || [ "$PGID" != "0" ]; then
+    groupmod -o -g "$PGID" abc 2>/dev/null || true
+    usermod  -o -u "$PUID" abc 2>/dev/null || true
+    mkdir -p /app/data
+    chown -R "$PUID:$PGID" /app/data 2>/dev/null || true
+    exec gosu abc "$@"
+else
+    exec "$@"
+fi

main.py

@@ -1,25 +1,74 @@
 import asyncio
 import json
 import logging
+import os
+import re
+from contextlib import asynccontextmanager
 
-from fastapi import BackgroundTasks, FastAPI, HTTPException, Request
+from fastapi import FastAPI, HTTPException, Request
 from fastapi.responses import HTMLResponse, StreamingResponse
 from fastapi.staticfiles import StaticFiles
 from fastapi.templating import Jinja2Templates
 from pydantic import BaseModel
 
-from arte_api import fetch_concerts, invalidate_cache, CATEGORIES
+from arte_api import fetch_concerts, get_all_concerts, get_concerts_by_category, invalidate_cache, CATEGORIES
 from downloader import DownloadManager
 
 logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
 
-app = FastAPI(title="Arte-dl")
+AUTO_DL_INTERVAL = int(os.getenv("AUTO_DL_INTERVAL", "3600"))
-app.mount("/static", StaticFiles(directory="static"), name="static")
-templates = Jinja2Templates(directory="templates")
 
 dm = DownloadManager()
 
 
+async def _run_auto_dl_check() -> int:
+    cats = dm.get_watched_categories()
+    if not cats:
+        return 0
+    total = 0
+    for cat in cats:
+        concerts = await get_concerts_by_category(cat)
+        for c in concerts:
+            if not dm.already_enqueued(c["url"]):
+                m = re.search(r"\b(20\d{2})\b", c.get("subtitle", ""))
+                year = int(m.group(1)) if m else c.get("tmdb_year")
+                await dm.enqueue(c["url"], c["title"], c.get("subtitle", ""), year, cat)
+                total += 1
+    return total
+
+
+async def _auto_dl_loop():
+    await asyncio.sleep(60)
+    while True:
+        try:
+            n = await _run_auto_dl_check()
+            if n:
+                logger.info("Auto-DL: enqueued %d new concert(s)", n)
+        except Exception as e:
+            logger.warning("Auto-DL check failed: %s", e)
+        await asyncio.sleep(AUTO_DL_INTERVAL)
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    await dm.resume_pending()
+    tasks = [
+        asyncio.create_task(dm.start_worker()),
+        asyncio.create_task(_auto_dl_loop()),
+        asyncio.create_task(get_all_concerts()),   # pre-warm cache at startup
+    ]
+    yield
+    for t in tasks:
+        t.cancel()
+    await asyncio.gather(*tasks, return_exceptions=True)
+
+
+app = FastAPI(title="Arte-dl", lifespan=lifespan)
+app.mount("/static", StaticFiles(directory="static"), name="static")
+templates = Jinja2Templates(directory="templates")
+
+
 # ------------------------------------------------------------------ pages
 
 
@@ -41,12 +90,46 @@ async def api_concerts(page: int = 1, search: str = "", page_size: int = 24, cat
     return await fetch_concerts(page=page, search=search, page_size=page_size, category=category)
 
 
+@app.get("/api/cache-ts")
+async def api_cache_ts():
+    from arte_api import _cache
+    return {"ts": _cache["ts"], "count": len(_cache["data"])}
+
+
 @app.post("/api/refresh")
 async def api_refresh():
     count = await invalidate_cache()
     return {"count": count}
 
 
+# ------------------------------------------------------------------ API: auto-download
+
+
+@app.get("/api/auto-dl")
+async def api_auto_dl_list():
+    return dm.get_watched_categories()
+
+
+@app.post("/api/auto-dl/check")
+async def api_auto_dl_check():
+    n = await _run_auto_dl_check()
+    return {"enqueued": n}
+
+
+@app.post("/api/auto-dl/{category}")
+async def api_auto_dl_watch(category: str):
+    if category not in CATEGORIES:
+        raise HTTPException(status_code=404, detail="Unknown category")
+    dm.watch_category(category)
+    return {"watched": True}
+
+
+@app.delete("/api/auto-dl/{category}")
+async def api_auto_dl_unwatch(category: str):
+    dm.unwatch_category(category)
+    return {"watched": False}
+
+
 # ------------------------------------------------------------------ API: downloads
 
 
@@ -55,13 +138,14 @@ class DownloadRequest(BaseModel):
     title: str
     subtitle: str = ""
     year: int | None = None
+    category: str = ""
 
 
 @app.post("/api/download")
-async def api_download(req: DownloadRequest, bg: BackgroundTasks):
+async def api_download(req: DownloadRequest):
     if not req.url:
         raise HTTPException(status_code=400, detail="url required")
-    dl_id = dm.enqueue(req.url, req.title, req.subtitle, req.year, bg)
+    dl_id = await dm.enqueue(req.url, req.title, req.subtitle, req.year, req.category)
     return {"id": dl_id}
 
 

static/app.js

@@ -50,18 +50,58 @@ function debounce(fn, ms) {
 // ── Categories ───────────────────────────────────────────────────────────────
 async function loadCategories() {
   try {
-    const cats = await fetch('/api/categories').then(r => r.json());
+    const [cats, watched] = await Promise.all([
+      fetch('/api/categories').then(r => r.json()),
+      fetch('/api/auto-dl').then(r => r.json()),
+    ]);
+    const watchedSet = new Set(watched);
     cats.forEach(cat => {
       const btn = document.createElement('button');
       btn.className = 'cat-pill';
       btn.dataset.cat = cat;
-      btn.textContent = cat;
+
+      const label = document.createElement('span');
+      label.textContent = cat;
+      btn.appendChild(label);
+
+      const icon = document.createElement('span');
+      icon.className = 'auto-icon' + (watchedSet.has(cat) ? ' active' : '');
+      icon.title = watchedSet.has(cat) ? 'Auto-DL actif — cliquer pour désactiver' : 'Activer le téléchargement automatique';
+      icon.dataset.cat = cat;
+      icon.textContent = '⬇';
+      btn.appendChild(icon);
+
       catBar.appendChild(btn);
     });
   } catch {}
 }
 
+async function toggleAutoDl(cat, iconEl) {
+  const isActive = iconEl.classList.contains('active');
+  try {
+    await fetch(`/api/auto-dl/${encodeURIComponent(cat)}`, { method: isActive ? 'DELETE' : 'POST' });
+    iconEl.classList.toggle('active', !isActive);
+    iconEl.title = !isActive ? 'Auto-DL actif — cliquer pour désactiver' : 'Activer le téléchargement automatique';
+
+    if (!isActive) {
+      statusText.textContent = `Auto-DL ${cat} : vérification en cours…`;
+      const { enqueued } = await fetch('/api/auto-dl/check', { method: 'POST' }).then(r => r.json());
+      statusText.textContent = enqueued > 0
+        ? `Auto-DL ${cat} : ${enqueued} concert${enqueued > 1 ? 's' : ''} ajouté${enqueued > 1 ? 's' : ''} à la file.`
+        : `Auto-DL ${cat} activé — aucun nouveau concert pour l'instant.`;
+    } else {
+      statusText.textContent = `Auto-DL ${cat} désactivé.`;
+    }
+  } catch {}
+}
+
 catBar.addEventListener('click', e => {
+  const icon = e.target.closest('.auto-icon');
+  if (icon) {
+    e.stopPropagation();
+    toggleAutoDl(icon.dataset.cat, icon);
+    return;
+  }
   const pill = e.target.closest('.cat-pill');
   if (!pill) return;
   catBar.querySelectorAll('.cat-pill').forEach(p => p.classList.remove('active'));
@@ -110,9 +150,10 @@ function renderConcerts(data) {
     const dl  = state.downloadedUrls.has(c.url) ? `<span class="card-downloaded">✓ Téléchargé</span>` : '';
     const date = fmtDate(c.upload_date);
     const sub  = c.subtitle ? `<div class="card-subtitle">${c.subtitle}</div>` : '';
+    const downloadedClass = state.downloadedUrls.has(c.url) ? 'downloaded' : '';
 
     return `
-      <div class="card" data-id="${c.id}" tabindex="0" role="button" aria-label="${c.title}">
+      <div class="card ${downloadedClass}" data-id="${c.id}" tabindex="0" role="button" aria-label="${c.title}">
         <div class="card-thumb-wrap">
           ${thumb}${dur}${dl}
         </div>
@@ -209,18 +250,15 @@ function openModal(concert) {
 
   const btnDl = $('btn-download');
   const alreadyDone = state.downloadedUrls.has(concert.url);
-  btnDl.textContent = alreadyDone ? '✓ Déjà téléchargé' : 'Télécharger';
+  btnDl.textContent = alreadyDone ? 'Re-télécharger' : 'Télécharger';
-  btnDl.prepend((() => {
+  btnDl.classList.toggle('btn-redownload', alreadyDone);
-    const s = document.createElementNS('http://www.w3.org/2000/svg','svg');
+  const dlIcon = document.createElementNS('http://www.w3.org/2000/svg','svg');
-    s.setAttribute('viewBox','0 0 20 20'); s.setAttribute('fill','none');
+  dlIcon.setAttribute('viewBox','0 0 20 20'); dlIcon.setAttribute('fill','none');
-    s.setAttribute('width','16'); s.setAttribute('height','16');
+  dlIcon.setAttribute('width','16'); dlIcon.setAttribute('height','16');
-    if (!alreadyDone) {
+  dlIcon.innerHTML = `<path d="M10 3v9M7 9l3 3 3-3" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"/>
-      s.innerHTML = `<path d="M10 3v9M7 9l3 3 3-3" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"/>
+    <path d="M4 15h12" stroke="currentColor" stroke-width="1.8" stroke-linecap="round"/>`;
-        <path d="M4 15h12" stroke="currentColor" stroke-width="1.8" stroke-linecap="round"/>`;
+  btnDl.prepend(dlIcon);
-    }
+  btnDl.disabled = false;
-    return s;
-  })());
-  btnDl.disabled = alreadyDone;
 
   $('dl-progress-wrap').hidden = true;
   $('dl-progress-fill').style.width = '0%';
@@ -266,11 +304,11 @@ $('btn-download').addEventListener('click', async () => {
 
   try {
     const yearMatch = (c.subtitle || '').match(/\b(20\d{2})\b/);
-    const year = yearMatch ? parseInt(yearMatch[1]) : null;
+    const year = yearMatch ? parseInt(yearMatch[1]) : (c.tmdb_year || null);
     const res = await fetch('/api/download', {
       method: 'POST',
       headers: { 'Content-Type': 'application/json' },
-      body: JSON.stringify({ url: c.url, title: c.title, subtitle: c.subtitle || '', year }),
+      body: JSON.stringify({ url: c.url, title: c.title, subtitle: c.subtitle || '', year, category: state.category }),
     });
     const { id } = await res.json();
     trackDownload(id, c.title, c.url);
@@ -301,7 +339,10 @@ function trackDownload(id, title, url) {
       $('dl-progress-label').textContent = `${Math.round(pct)}%`;
       if (s.state === 'done') {
         $('dl-progress-label').textContent = '✓ Terminé';
-        $('btn-download').textContent = '✓ Téléchargé';
+        const b = $('btn-download');
+        b.textContent = 'Re-télécharger';
+        b.classList.add('btn-redownload');
+        b.disabled = false;
       }
     }
 
@@ -408,8 +449,25 @@ document.addEventListener('keydown', e => {
   }
 });
 
+// ── Cache auto-update polling ─────────────────────────────────────────────────
+let _cacheTs = 0;
+
+async function pollCacheTs() {
+  try {
+    const { ts, count } = await fetch('/api/cache-ts').then(r => r.json());
+    if (_cacheTs && ts > _cacheTs && count > 0) {
+      _cacheTs = ts;
+      await refresh();
+    } else if (!_cacheTs) {
+      _cacheTs = ts;
+    }
+  } catch {}
+}
+
 // ── Init ─────────────────────────────────────────────────────────────────────
 (async () => {
   await Promise.all([loadCategories(), refreshDlHistory()]);
   await refresh();
+  _cacheTs = (await fetch('/api/cache-ts').then(r => r.json()).catch(() => ({ts:0}))).ts;
+  setInterval(pollCacheTs, 30_000);
 })();

static/style.css

@@ -216,10 +216,13 @@ body {
   cursor: pointer;
   font-family: 'Inter', sans-serif;
   font-size: 13px;
-  padding: 6px 16px;
+  padding: 6px 6px 6px 14px;
   white-space: nowrap;
   transition: color var(--transition), border-color var(--transition), background var(--transition);
   flex-shrink: 0;
+  display: flex;
+  align-items: center;
+  gap: 4px;
 }
 
 .cat-pill:hover {
@@ -234,6 +237,37 @@ body {
   font-weight: 600;
 }
 
+.auto-icon {
+  font-size: 12px;
+  line-height: 1;
+  opacity: 0.55;
+  padding: 2px 5px;
+  border-radius: 10px;
+  transition: opacity var(--transition), background var(--transition), color var(--transition);
+  cursor: pointer;
+}
+
+.auto-icon:hover {
+  opacity: 1;
+  background: rgba(255,255,255,0.12);
+}
+
+.auto-icon.active {
+  opacity: 1;
+  color: var(--gold);
+}
+
+.cat-pill.active .auto-icon {
+  opacity: 0.6;
+  color: #000;
+}
+
+.cat-pill.active .auto-icon.active {
+  opacity: 1;
+  background: rgba(0,0,0,0.18);
+  color: #000;
+}
+
 /* ══ GRID ══════════════════════════════════════════════════════════════════ */
 .grid {
   display: grid;
@@ -252,7 +286,7 @@ body {
   border-radius: var(--radius);
   overflow: hidden;
   cursor: pointer;
-  transition: transform var(--transition), box-shadow var(--transition), border-color var(--transition);
+  transition: transform var(--transition), box-shadow var(--transition), border-color var(--transition), opacity var(--transition), filter var(--transition);
   position: relative;
 }
 
@@ -262,6 +296,19 @@ body {
   border-color: var(--gold-dim);
 }
 
+.card.downloaded {
+  opacity: 0.45;
+  filter: grayscale(0.6);
+}
+
+.card.downloaded:hover {
+  opacity: 0.65;
+  filter: grayscale(0.4);
+  transform: none;
+  box-shadow: none;
+  border-color: var(--border);
+}
+
 .card-thumb-wrap {
   position: relative;
   aspect-ratio: 16/9;
@@ -582,6 +629,8 @@ body {
 .btn-download:hover { opacity: 0.88; transform: translateY(-1px); }
 .btn-download:active { transform: none; }
 .btn-download:disabled { opacity: 0.5; cursor: default; transform: none; }
+.btn-download.btn-redownload { background: transparent; color: var(--text-muted); border: 1px solid var(--border); }
+.btn-download.btn-redownload:hover { color: var(--text); border-color: var(--text-muted); opacity: 1; }
 
 .btn-watch {
   color: var(--text-dim);

tmdb.py

@@ -11,7 +11,7 @@ logger = logging.getLogger(__name__)
 
 _CACHE_DAYS = 30
 _THRESHOLD  = 0.45
-_DB         = "arte_dl.db"
+_DB         = "data/arte_dl.db"
 _IMG_BASE   = "https://image.tmdb.org/t/p"
 _SEARCH_URL = "https://api.themoviedb.org/3/search/movie"
 
@@ -33,9 +33,14 @@ def _init_db():
                 tmdb_id   INTEGER,
                 poster    TEXT,
                 backdrop  TEXT,
+                year      INTEGER,
                 cached_at TEXT NOT NULL
             )
         """)
+        try:
+            conn.execute("ALTER TABLE tmdb_cache ADD COLUMN year INTEGER")
+        except Exception:
+            pass
 
 
 def _search(query: str) -> list[dict]:
@@ -72,39 +77,42 @@ def lookup(arte_id: str, title: str, subtitle: str) -> dict | None:
     cutoff = (datetime.now() - timedelta(days=_CACHE_DAYS)).isoformat()
     with sqlite3.connect(_DB) as conn:
         row = conn.execute(
-            "SELECT tmdb_id, poster, backdrop FROM tmdb_cache WHERE arte_id=? AND cached_at>?",
+            "SELECT tmdb_id, poster, backdrop, year FROM tmdb_cache WHERE arte_id=? AND cached_at>?",
             (arte_id, cutoff),
         ).fetchone()
 
     if row is not None:
-        tmdb_id, poster, backdrop = row
+        tmdb_id, poster, backdrop, year = row
         if tmdb_id is None:
             return None  # cached "no match"
-        return _build(tmdb_id, poster, backdrop)
+        return _build(tmdb_id, poster, backdrop, year)
 
     # Query TMDB
     query = f"{title} {subtitle}".strip()
     results = _search(query) or (_search(title) if subtitle else [])
     match = _best_match(results, title, subtitle)
 
-    tmdb_id   = match["id"]            if match else None
+    tmdb_id   = match["id"]                                       if match else None
-    poster    = match.get("poster_path")    if match else None
+    poster    = match.get("poster_path")                          if match else None
-    backdrop  = match.get("backdrop_path")  if match else None
+    backdrop  = match.get("backdrop_path")                        if match else None
+    rd        = (match.get("release_date") or "")[:4]            if match else ""
+    year      = int(rd) if rd.isdigit() else None
 
     with sqlite3.connect(_DB) as conn:
         conn.execute(
-            "INSERT OR REPLACE INTO tmdb_cache VALUES (?,?,?,?,?)",
+            "INSERT OR REPLACE INTO tmdb_cache (arte_id, tmdb_id, poster, backdrop, year, cached_at) VALUES (?,?,?,?,?,?)",
-            (arte_id, tmdb_id, poster, backdrop, datetime.now().isoformat()),
+            (arte_id, tmdb_id, poster, backdrop, year, datetime.now().isoformat()),
         )
 
-    return _build(tmdb_id, poster, backdrop) if tmdb_id else None
+    return _build(tmdb_id, poster, backdrop, year) if tmdb_id else None
 
 
-def _build(tmdb_id: int, poster: str | None, backdrop: str | None) -> dict:
+def _build(tmdb_id: int, poster: str | None, backdrop: str | None, year: int | None = None) -> dict:
     return {
-        "tmdb_id":   tmdb_id,
+        "tmdb_id":       tmdb_id,
         "tmdb_poster":   f"{_IMG_BASE}/w500{poster}"   if poster   else None,
         "tmdb_backdrop": f"{_IMG_BASE}/w1280{backdrop}" if backdrop else None,
+        "tmdb_year":     year,
     }