dcosnet
/
AutoIngest
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
AutoIngest/README.md

13 KiB
Raw Blame History

AutoIngest

A Windows tray-resident photo importer for ecommerce sellers. Jeremy Anderson · dcos.net · info@dcos.net Version 3.1 · MIT licensed

What it does

Plug in a camera, SD card, USB reader, or phone. AutoIngest moves the photos into a dated folder (Pictures\YYYY-MM-DD\), converts any non-JPG format to progressive JPG, applies your branding, and gets out of the way. Built for a seller who photographs items and needs JPGs in a folder with zero clicks.

The four pillars:

  1. Import. Photos move off the device into ~/Pictures/<today>/. JPGs copy straight through; RAW/HEIC/PNG convert to JPG via Magick.NET. Same-day dedupe (size then SHA-256) skips re-imports.
  2. Trust. An opt-in device registry decides what imports. A friend's USB stick or phone is never touched. Each device is fingerprinted by volume serial (SD/USB) or PTP serial (phone) — never by drive letter.
  3. Brand. A welcome wizard on first launch sets up your store identity: a text or logo watermark burned into every converted photo, plus EXIF copyright that survives metadata stripping.
  4. Maintain. Optional retention sweeps dated folders older than a threshold to the Recycle Bin. Optional autostart installs the exe to a stable home and launches it at login.

Features

Import pipeline

  • Polls mass-storage drives (kernel32 GetDriveType / GetVolumeInformationW) and MTP phones (Shell.Application COM on an STA thread) every 4 seconds
  • Mass storage: probes DCIM / Pictures / root / 3-level deep scan
  • MTP: walks the device shell namespace for DCIM-equivalent folders
  • JPG/JPEG files copy through; everything else converts via Magick.NET (30+ formats: RAW, HEIC, PNG, TIFF, WebP, SVG, …)
  • Output is progressive, optimized JPG at native resolution
  • Verify-before-delete: every source is reclaimed only after the destination is confirmed present, correctly sized, and (for converts) re-decoded
  • Same-day duplicate aware: byte-identical files skipped; name collisions with different content get _1, _2 suffixes (capped at 9999, GUID fallback)
  • Single-instance enforcement via named mutex

Device registry (opt-in trust)

  • First time a device connects, choose Register & Import, Import Once, or Ignore
  • Registered devices are keyed by a stable fingerprint:
    • SD/USB → volume serial from GetVolumeInformationW
    • Phone → PTP/MTP serial via the shell namespace
    • No serial available → weak fallback (label + size), flagged in the UI
  • A friend's unregistered device is silently skipped (or prompts, per your setting)
  • Manage from the tray: Settings… → Registered devices

Image quality toggles (Options ▸)

  • Auto-orient (default on) — honors the EXIF orientation tag by rotating the pixels, so phone photos do not land sideways in browser uploads
  • Strip metadata (default on) — removes EXIF/XMP/GPS on export so listing photos do not leak the seller's location
  • Delete from phone after import (default off) — phones only; mirrors the move behavior SD/USB have always had. SD/USB always move/convert as before

Branding (optional, off by default)

Set up via the first-launch welcome wizard, or later from Options ▸ Branding ▸ Edit branding…:

  • Text watermark — store name / handle / URL burned onto the photo (position + opacity configurable, live preview)
  • Logo watermark — PNG overlaid in a corner (scale + opacity configurable)
  • EXIF copyright — Artist / Copyright / Description embedded as the only surviving metadata: strip runs first (camera/GPS/software tags gone), then your copyright writes back
  • Brand tray icon — the tray icon renders your logo or store initial on a rounded tile whose background auto-contrasts the mark (light mark → dark tile, dark mark → light tile)
  • Applies to converted files only (same scope as auto-orient / strip)

Photo retention (optional, off by default)

  • Dated folders older than a chosen age go to the Recycle Bin on a daily sweep (and once on startup)
  • Age is measured by most recent activity across the folder's files — access a file and the folder's clock resets
  • Presets: 1 week, 1 month, 3 months, 6 months, 12 months, 2 years, 7 years
  • Safety rails: today's folder is exempt; a 10-minute freshness guard protects active imports against midnight rollover; non-dated folders are never touched; everything is recoverable
  • Access-tracking aware: detects NTFS LastAccessTime state and falls back to import date when Windows disables access updates (the default)

Start with Windows (optional toggle)

  • Options ▸ Start with Windows copies the exe to %LocalAppData%\Programs\AutoIngest\ and places a shortcut in the user's Startup folder (shell:startup)
  • No admin rights, no UAC, no registry entry. Per-user. Takes effect at next login
  • The shortcut's existence is the source of truth — the checkbox reflects reality, refreshed every time the menu opens

Toast UI

  • Lives in the system tray by default — no window at startup
  • A borderless toast pops into the bottom-right corner during an import (or when an unknown device connects)
  • Live status line (Moving 3/12: IMG_004.jpg) and a scrolling log
  • Auto-hides ~4 seconds after the import completes
  • Left-click the tray icon to peek; right-click for the menu

Architecture

AutoIngest/
├── AutoIngest.csproj              .NET 8 / WinForms / single NuGet dependency
├── App/                           UI layer (namespace AutoIngest.App)
│   ├── Program.cs                 Entry point, single-instance mutex
│   ├── MainForm.cs                Tray icon, toast, coordinator wiring, Options menu
│   ├── MainForm.Designer.cs       Toast popup layout (WinForms Designer)
│   ├── MainForm.resx              Designer resource header
│   ├── SettingsForm.cs            Settings: device registry + retention + access tracking
│   ├── WelcomeForm.cs             First-run welcome / branding wizard (new installs only)
│   ├── BrandingForm.cs            Branding editor opened from the tray
│   ├── BrandingEditorPanel.cs     Shared branding fields + live preview
│   ├── TrayIconFactory.cs         Builds the tray icon as a micro brand mark
│   ├── AutostartManager.cs        Self-install + Startup-folder shortcut (IShellLinkW COM)
│   ├── Theme.cs                   Shared dark-theme palette
│   └── app.manifest               Windows 10/11 compatibility manifest
├── Core/                          Config/models layer (namespace AutoIngest.Core)
│   ├── ConfigManager.cs           JSON config persistence (atomic write, .bak fallback)
│   ├── DeviceRegistry.cs          In-memory registry of trusted devices
│   └── FileLogger.cs              Rolling persistent log at %LocalAppData%\AutoIngest\
└── Engine/                        Import/conversion layer (namespace AutoIngest.Engine)
    ├── IDeviceSource.cs           Interface for a source of connected devices
    ├── DeviceIdentity.cs          Connected-device DTO
    ├── IPhotoProvider.cs          Interface for enumerating + removing source photos
    ├── SDCardMonitor.cs           Mass-storage source (SD/USB) — volume-serial ids
    ├── MtpDeviceSource.cs         Phone source — Shell.Application COM on an STA thread
    ├── DeviceImporter.cs          Shared photo-discovery + move/convert pipeline
    ├── ImportCoordinator.cs       Polling loop, registry gate, retention sweep, event fan-out
    ├── FolderRetentionPolicy.cs   Enumerates dated folders + sweeps aged ones to Recycle Bin
    ├── AccessTimeTracker.cs       Detects/enables NTFS last-access-time tracking
    ├── BrandingRenderer.cs        Burns text/logo watermarks into a photo
    ├── FileSize.cs                Shared human-readable byte formatting
    └── ImageConverter.cs          Magick.NET wrapper: orient → brand → strip → copyright → JPG

Single NuGet dependency: Magick.NET-Q16-AnyCPU. Everything else (drive detection, shell COM, recycle bin, shortcut creation, access-time probing) uses OS-provided APIs.

Layering: App → Core, App → Engine, Engine → Core. No reverse references. Each class owns one responsibility.

Design decisions

Decision Rationale
Move, not copy The card is the source of truth; duplicates after import cause confusion. Move once, done.
Copy-and-leave for phones by default Phones are not a filesystem you can File.Move from; auto-deleting from a phone is risky. Opt-in toggle mirrors the move behavior.
Opt-in device registry A friend's USB stick or phone must never auto-import. Each device is approved once and remembered by serial, not drive letter.
Strip-everything-then-write-branding Camera model, GPS, and software tags are wiped; the seller's copyright writes back as the sole surviving metadata. Privacy and attribution in one pipeline.
Retention off by default Auto-deleting photos is dangerous. The seller picks a threshold; nothing happens until they do.
Recycle Bin, not permanent delete Mistakes recover. Aged folders go to the Recycle Bin.
Filesystem timestamps are the access list Any access resets a folder's clock. No parallel skip-list to maintain or drift.
Honest access-tracking detection Windows disables last-access times by default; we detect it, explain it, and offer to enable it.
Startup-folder shortcut The classic autorun mechanism, fully supported on Windows 11. No admin, no UAC.
%LocalAppData%\Programs install target User-writable, per-user, survives reboots. The system root (C:\Windows\) is admin-owned and Defender-scrutinized.
C# / .NET 8 over Python Single-exe deployment. No runtime or pip on the target PC.
WinForms over WPF Direct control layout, no XAML complexity.
Magick.NET over System.Drawing One library handles all 30+ formats including RAW, HEIC, SVG.
Polling (4 s) over WMI events Deterministic, no COM apartment threading, consistent across Windows versions.
Flat output directory Matches the workflow: pull to date folder, open in browser, attach to listing.

Building

Prerequisites

  • Windows 10/11
  • Visual Studio 2022 with the .NET desktop development workload, or the .NET 8 SDK standalone

Visual Studio

  1. Open AutoIngest.sln
  2. Build → Build Solution (Ctrl+Shift+B)
  3. Output: AutoIngest/bin/Debug/net8.0-windows/AutoIngest.exe

Command line

cd AutoIngest
dotnet restore
dotnet build -c Release

Publish a single exe

REM Framework-dependent (~15 MB, requires .NET 8 Desktop Runtime on target)
publish.bat

REM Self-contained (~150-200 MB, runs on any Windows 10/11 PC)
publish-standalone.bat

See QUICKSTART.md for the 5-minute end-to-end flow.

Deployment

Copy AutoIngest.exe to the target PC. No installer required.

  • Framework-dependent build requires the .NET 8 Desktop Runtime. Windows 11 includes it; Windows 10 may need it from dotnet.microsoft.com.
  • Self-contained build has no prerequisites. Runs on any Windows 10/11 PC.

Configuration

Config file: ~/.autoingest_config.json — created on first run. Fields:

Field Default Purpose
JpgQuality 92 Output JPG quality
AutoImport true Whether the monitor runs on startup
PromptOnUnknown true Whether an unregistered device prompts vs. is silently skipped
Devices [] Opt-in registry: stable id, name, kind, added date
RetentionEnabled false Aged-folder auto-recycle
RetentionDays 180 Age threshold in days
AutoOrient true EXIF orientation correction
StripExif true Strip EXIF/GPS/XMP metadata
DeleteFromPhoneAfterImport false Phone-side delete after verified copy (phones only)
SetupCompleted true Welcome-wizard gating marker
Branding (object) Enabled flag, store identity, watermark settings, EXIF copyright

The registry, retention, and branding are editable from the UI (tray → Settings… and Options ▸ Branding ▸ Edit branding…). Hand-editing the JSON is supported but unnecessary.

Tech stack

Component Technology
Language C# 12 / .NET 8
UI framework Windows Forms
Image processing Magick.NET-Q16-AnyCPU (ImageMagick 7)
Mass-storage detection kernel32 GetDriveType + GetVolumeInformationW via P/Invoke
Phone (MTP) detection Shell.Application COM interop on an STA thread
Recycle Bin Microsoft.VisualBasic.FileIO (BCL)
Shortcut creation IShellLinkW + IPersistFile COM interop
Access-time probe fsutil via System.Diagnostics.Process
Serialization System.Text.Json
Target Windows 10 1809+ / Windows 11

License

MIT License. See LICENSE.