ParkingRobot/docs/superpowers/plans/2026-08-03-em-planner-osqp-backend-implementation.md

# EM Planner OSQP Backend Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Provide a solver-independent convex-QP contract and a pinned Windows x64 OSQP 1.0.0 backend that loads `osqp.dll` safely from the plugin directory.

**Architecture:** Mathematical planners build validated immutable CSC problems against `IQpSolver`; the OSQP adapter owns all native memory and maps native outcomes into planner-neutral statuses. The upstream shared library is built with a fixed ABI configuration, preloaded by absolute path, version-checked, and never allowed to crash the host when absent or incompatible.

**Tech Stack:** C# 10, .NET Standard 2.0, P/Invoke with Cdecl, OSQP 1.0.0 C API, builtin QDLDL algebra, CMake 3.18+, Visual Studio x64 compiler.

## Global Constraints

- This plan depends on `2026-08-03-em-planner-foundation-implementation.md` Task 1 and its verification host.
- Pin upstream source tag `v1.0.0`; do not bind the incompatible 0.6 API.
- Build Windows x64, double precision, 32-bit indices, unpacked settings, builtin algebra, shared library, no MKL or CUDA.
- Required build switches: `OSQP_USE_FLOAT=OFF`, `OSQP_USE_LONG=OFF`, `OSQP_PACK_SETTINGS=OFF`, `OSQP_ALGEBRA_BACKEND=builtin`, `OSQP_BUILD_SHARED_LIB=ON`.
- Native library filename in source and deployed plugin is exactly `osqp.dll`.
- `ClumsyPilot.dll` locates and preloads the sibling DLL from `Assembly.Location`; current directory and system `PATH` are not inputs.
- Every native entry point uses `CallingConvention.Cdecl`.
- All pinned arrays, CSC wrappers, settings, and solver handles are released in reverse acquisition order.
- `SolvedInaccurate` is publishable only after independent strict residual and domain validation.
- Missing DLL, wrong architecture, version mismatch, invalid exports, and setup failure return structured solver outcomes.
- Native verbose output is disabled.

---

## Locked File Structure

```text
ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/
├── IQpSolver.cs
├── QpSolveResult.cs
├── QpSolveStatus.cs
├── QpSolverSettings.cs
├── QuadraticProgram.cs
├── SparseCscMatrix.cs
├── SparseTripletBuilder.cs
└── Osqp/
    ├── OsqpNativeLoader.cs
    ├── OsqpNativeMethods.cs
    ├── OsqpNativeSolver.cs
    ├── OsqpNativeStructures.cs
    └── OsqpStatusMapper.cs

ClumsyPilot/ThirdParty/OSQP/
├── build-win-x64.ps1
├── LICENSE
├── NOTICE
├── VERSION
├── SHA256SUMS
└── win-x64/osqp.dll

ClumsyPilot/tests/EMPlannerVerificationHost/
├── OptimizationChecks.cs
└── OsqpChecks.cs
```

## Shared Interfaces

```csharp
public interface IQpSolver
{
    QpSolveResult Solve(QuadraticProgram problem, QpSolverSettings settings,
        IReadOnlyList<double> warmStart, CancellationToken cancellationToken);
}

public sealed class QuadraticProgram
{
    public QuadraticProgram(SparseCscMatrix upperTriangularP, IReadOnlyList<double> q,
        SparseCscMatrix a, IReadOnlyList<double> lowerBounds,
        IReadOnlyList<double> upperBounds);
    public int VariableCount { get; }
    public int ConstraintCount { get; }
}

public sealed class QpSolveResult
{
    public QpSolveStatus Status { get; }
    public IReadOnlyList<double> Primal { get; }
    public double Objective { get; }
    public double PrimalResidual { get; }
    public double DualResidual { get; }
    public int Iterations { get; }
    public TimeSpan SolveTime { get; }
    public string NativeStatus { get; }
    public string Diagnostic { get; }
}
```

Official references used to lock this ABI:

- `https://osqp.org/docs/interfaces/C.html`
- `https://osqp.org/docs/get_started/migration_guide.html`
- `https://github.com/osqp/osqp/tree/v1.0.0`

### Task 1: Solver-Neutral Sparse QP Contracts

**Files:**
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/IQpSolver.cs`
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/QpSolveResult.cs`
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/QpSolveStatus.cs`
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/QpSolverSettings.cs`
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/QuadraticProgram.cs`
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/SparseCscMatrix.cs`
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/SparseTripletBuilder.cs`
- Create: `ClumsyPilot/tests/EMPlannerVerificationHost/OptimizationChecks.cs`
- Modify: `ClumsyPilot/tests/EMPlannerVerificationHost/Program.cs`

**Interfaces:**
- Consumes: `NumericGuard` and cancellation tokens.
- Produces: the shared interfaces above and deterministic sparse-matrix assembly used by LS and ST.

- [ ] **Step 1: Write failing CSC canonicalization checks**

Build triplets in shuffled order with duplicate coordinates and assert the resulting CSC matrix:

```text
has ColumnPointers length ColumnCount+1
sorts row indices ascending inside each column
sums duplicate coordinates
drops exact zero sums
rejects NaN, infinity, negative indices, and out-of-range indices
stores only the upper triangle for P
```

Also construct the micro problem `min 0.5*x^2 - 2*x` subject to `0 <= x <= 1` and assert its immutable arrays cannot be changed through the source lists.

- [ ] **Step 2: Run and verify solver contracts are absent**

```powershell
dotnet run --project ClumsyPilot/tests/EMPlannerVerificationHost/EMPlannerVerificationHost.csproj -- optimization
```

Expected: build failure naming `SparseTripletBuilder`.

- [ ] **Step 3: Implement canonical CSC and QP validation**

`SparseCscMatrix` stores copied arrays `Values`, `RowIndices`, and `ColumnPointers`. Validate monotonic pointers, `ColumnPointers[0]==0`, final pointer equals nonzero count, and all rows are in range. `QuadraticProgram` enforces square P, matching variable dimensions, matching constraint dimensions, `lower<=upper`, finite coefficients, and bounds limited to `±1e30` rather than CLR infinity.

Use these exact statuses:

```csharp
public enum QpSolveStatus
{
    Solved,
    SolvedInaccurate,
    PrimalInfeasible,
    DualInfeasible,
    MaximumIterations,
    TimeLimit,
    Cancelled,
    SolverUnavailable,
    InvalidProblem,
    NativeError
}
```

- [ ] **Step 4: Run optimization checks**

Expected: `PASS optimization`.

- [ ] **Step 5: Commit QP contracts**

```powershell
git add ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization ClumsyPilot/tests/EMPlannerVerificationHost
git commit -m "feat: add solver-neutral QP contracts"
```

### Task 2: Reproducible OSQP 1.0.0 Native Package

**Files:**
- Create: all files under `ClumsyPilot/ThirdParty/OSQP/`

**Interfaces:**
- Consumes: Git, CMake 3.18+, and a Visual Studio x64 compiler.
- Produces: a versioned `win-x64/osqp.dll` with a recorded SHA-256 and matching license files.

- [ ] **Step 1: Write the native build script**

`build-win-x64.ps1` must create a unique temporary directory, clone only tag `v1.0.0`, configure with this exact command shape, and remove the temporary directory in `finally`:

```powershell
cmake -S $sourceRoot -B $buildRoot -A x64 `
  -DOSQP_ALGEBRA_BACKEND=builtin `
  -DOSQP_BUILD_SHARED_LIB=ON `
  -DOSQP_BUILD_STATIC_LIB=OFF `
  -DOSQP_BUILD_DEMO_EXE=OFF `
  -DOSQP_BUILD_UNITTESTS=OFF `
  -DOSQP_USE_FLOAT=OFF `
  -DOSQP_USE_LONG=OFF `
  -DOSQP_PACK_SETTINGS=OFF `
  -DOSQP_ENABLE_PRINTING=OFF `
  -DOSQP_CODEGEN=OFF `
  -DOSQP_ENABLE_DERIVATIVES=OFF
cmake --build $buildRoot --config Release --target osqp
```

The script resolves the generated DLL explicitly, verifies exactly one match, copies upstream `LICENSE` and `NOTICE`, writes `VERSION` with tag and build flags, computes `Get-FileHash -Algorithm SHA256`, and writes `SHA256SUMS` using a lowercase hexadecimal digest.

- [ ] **Step 2: Execute the build script**

```powershell
powershell -ExecutionPolicy Bypass -File ClumsyPilot/ThirdParty/OSQP/build-win-x64.ps1
```

Expected: `win-x64/osqp.dll`, `LICENSE`, `NOTICE`, `VERSION`, and `SHA256SUMS` exist; the script prints `OSQP v1.0.0 win-x64 package ready`.

- [ ] **Step 3: Verify architecture, exports, and hash**

Use `dumpbin /headers` to assert machine `x64`, `dumpbin /exports` to assert `osqp_version`, `osqp_setup`, `osqp_solve`, and `osqp_cleanup`, then recompute SHA-256 and compare with `SHA256SUMS`. A missing tool is a failed packaging gate, not a skipped check.

- [ ] **Step 4: Verify license contents came from the pinned tag**

Compare bytes against the tag checkout before the temporary checkout is removed. Expected: exact equality for both files.

- [ ] **Step 5: Commit the reproducible native package**

```powershell
git add ClumsyPilot/ThirdParty/OSQP
git commit -m "build: pin OSQP 1.0.0 win-x64"
```

### Task 3: Absolute-Path Native Loader and ABI Structures

**Files:**
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpNativeLoader.cs`
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpNativeMethods.cs`
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpNativeStructures.cs`
- Create: `ClumsyPilot/tests/EMPlannerVerificationHost/OsqpChecks.cs`
- Modify: `ClumsyPilot/tests/EMPlannerVerificationHost/Program.cs`

**Interfaces:**
- Consumes: the pinned DLL and Task 1 QP contracts.
- Produces: a process-lifetime loader plus exact OSQP 1.0.0 double/int32 structures.

- [ ] **Step 1: Write failing loader checks**

Copy the verification host output to a temporary `plugins` directory with `ClumsyPilot.dll` and, in separate cases, no native DLL, a text file named `osqp.dll`, and the real DLL. Assert the first two return `SolverUnavailable` diagnostics without `BadImageFormatException` escaping; the real DLL reports version `1.0.0`. Start 16 parallel first-use calls and assert a single stable module handle.

- [ ] **Step 2: Run the OSQP group without loader implementation**

Expected: build failure naming `OsqpNativeLoader`.

- [ ] **Step 3: Implement loader and ABI definitions**

Use Windows `LoadLibraryW`, `GetProcAddress`, and `FreeLibrary` from `kernel32`; resolve the plugin directory from `typeof(OsqpNativeLoader).Assembly.Location`. Reject `IntPtr.Size != 8`. Preload the absolute sibling path and retain the handle for process lifetime.

Define `OSQPInt` as C# `int` and `OSQPFloat` as C# `double`, matching the pinned build. Define sequential layouts for `OSQPCscMatrix`, `OSQPSettings`, `OSQPInfo`, `OSQPSolution`, and the four-pointer prefix of `OSQPSolver` exactly as the v1.0.0 public headers specify. Add an internal layout check for expected offsets and total sizes before the first solve.

Declare only these native functions initially:

```text
osqp_version
osqp_set_default_settings
osqp_setup
osqp_warm_start
osqp_solve
osqp_cleanup
```

Do not depend on `OSQPCscMatrix_new`, `OSQPCscMatrix_free`, `OSQPSettings_new`, or `OSQPSettings_free`: those helpers are not marked with the public export macro in the pinned header. Allocate the two CSC structures and settings block with `Marshal.AllocHGlobal`, initialize settings through `osqp_set_default_settings`, and release those managed-owned blocks with `Marshal.FreeHGlobal`.

- [ ] **Step 4: Run missing, corrupt, real, and concurrent loader checks**

Expected: `PASS osqp-loader` and no process crash.

- [ ] **Step 5: Commit loader and structures**

```powershell
git add ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp ClumsyPilot/tests/EMPlannerVerificationHost
git commit -m "feat: load pinned OSQP native library"
```

### Task 4: OSQP Solve Lifecycle and Status Mapping

**Files:**
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpNativeSolver.cs`
- Create: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpStatusMapper.cs`
- Modify: `ClumsyPilot/tests/EMPlannerVerificationHost/OsqpChecks.cs`

**Interfaces:**
- Consumes: `IQpSolver`, Task 3 native functions, and `QpSolverSettings`.
- Produces: `OsqpNativeSolver : IQpSolver`.

- [ ] **Step 1: Write failing solve/status checks**

Test three fixed QPs:

```text
bounded optimum: min 0.5*x^2 - 2*x, 0<=x<=1, expected x=1
equality optimum: min x^2+y^2, x+y=1, expected x=y=0.5
infeasible: x>=1 and x<=0, expected PrimalInfeasible
```

Assert residuals, iteration count, objective, native status, and solve time are populated. Add a `1e-9 second` time-limit case that maps only to `TimeLimit` or a valid solved status; no native status may be silently treated as solved.

- [ ] **Step 2: Run and verify `OsqpNativeSolver` is absent**

Expected: build failure naming `OsqpNativeSolver`.

- [ ] **Step 3: Implement one-shot native ownership**

Pin P/Q/A/L/U and optional warm-start arrays; allocate and populate P/A `OSQPCscMatrix` blocks; allocate settings and initialize it through `osqp_set_default_settings`; overwrite `verbose=0`, `warm_starting`, `polishing`, `max_iter`, `eps_abs`, `eps_rel`, and `time_limit`; call setup, optional warm start, solve, then marshal solution and info. Copy all result values before cleanup. Release the solver through `osqp_cleanup`, then settings/matrix blocks through `Marshal.FreeHGlobal`, then array pins in reverse order inside `finally`.

Map native status values exactly:

```text
1 Solved
2 SolvedInaccurate
3/4 PrimalInfeasible
5/6 DualInfeasible
7 MaximumIterations
8 TimeLimit
9/10/11 NativeError
```

Cancellation is checked before native setup and after solve. OSQP's configured time limit is the bound for a solve already inside native code.

- [ ] **Step 4: Run all OSQP checks repeatedly**

```powershell
1..20 | ForEach-Object {
  dotnet run --project ClumsyPilot/tests/EMPlannerVerificationHost/EMPlannerVerificationHost.csproj -- osqp
  if ($LASTEXITCODE -ne 0) { throw "OSQP verification failed on iteration $_" }
}
```

Expected: every iteration prints `PASS osqp-loader` and `PASS osqp-solve`.

- [ ] **Step 5: Commit solver lifecycle**

```powershell
git add ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp ClumsyPilot/tests/EMPlannerVerificationHost/OsqpChecks.cs
git commit -m "feat: solve QPs through OSQP"
```

### Task 5: Backend Completion Gate

**Files:**
- Modify: `ClumsyPilot/ParkrobTrajplanner/EMPlanner/README.md`
- Modify: `ClumsyPilot/tests/EMPlannerVerificationHost/Program.cs`

**Interfaces:**
- Consumes: all tasks in this plan.
- Produces: the stable `IQpSolver` boundary required by LS and ST plans.

- [ ] **Step 1: Document native deployment and diagnostics**

Add the exact source/deployment layouts, pinned version, build flags, license placement, absolute loading rule, and solver status mapping to the README.

- [ ] **Step 2: Run optimization and OSQP gates**

```powershell
dotnet run --project ClumsyPilot/tests/EMPlannerVerificationHost/EMPlannerVerificationHost.csproj -- optimization
dotnet run --project ClumsyPilot/tests/EMPlannerVerificationHost/EMPlannerVerificationHost.csproj -- osqp
git diff --check
```

Expected: both groups pass and Git reports no whitespace errors.

- [ ] **Step 3: Verify the DLL is self-contained**

Run a dependency inspection on `win-x64/osqp.dll`. Expected: only Windows system/runtime DLLs; no MKL, CUDA, or separately deployed QDLDL DLL.

- [ ] **Step 4: Verify clean plugin-directory loading**

Copy only `ClumsyPilot.dll` and `osqp.dll` to a fresh directory, copy the verification host executable beside them, and run the micro QP with the working directory set elsewhere. Expected: solved result, proving loading does not depend on current directory.

- [ ] **Step 5: Commit backend documentation**

```powershell
git add ClumsyPilot/ParkrobTrajplanner/EMPlanner/README.md ClumsyPilot/tests/EMPlannerVerificationHost/Program.cs
git commit -m "docs: describe OSQP plugin deployment"
```

## Completion Gate

- `IQpSolver` contains no OSQP-specific type.
- Pinned native metadata, license, notice, hash, and DLL agree with OSQP v1.0.0.
- Loader failures are structured and never terminate the host.
- Fixed feasible and infeasible QPs map to the correct statuses with finite diagnostics.
- Twenty repeated solve/cleanup cycles pass without handle growth or access violations.