ParkingRobot/docs/superpowers/plans/2026-08-03-em-planner-osqp-backend-implementation.md

EM Planner OSQP Backend Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Provide a solver-independent convex-QP contract and a pinned Windows x64 OSQP 1.0.0 backend that loads osqp.dll safely from the plugin directory.

Architecture: Mathematical planners build validated immutable CSC problems against IQpSolver; the OSQP adapter owns all native memory and maps native outcomes into planner-neutral statuses. The upstream shared library is built with a fixed ABI configuration, preloaded by absolute path, version-checked, and never allowed to crash the host when absent or incompatible.

Tech Stack: C# 10, .NET Standard 2.0, P/Invoke with Cdecl, OSQP 1.0.0 C API, builtin QDLDL algebra, CMake 3.18+, Visual Studio x64 compiler.

Global Constraints

• This plan depends on 2026-08-03-em-planner-foundation-implementation.md Task 1 and its verification host.
• Pin upstream source tag v1.0.0; do not bind the incompatible 0.6 API.
• Build Windows x64, double precision, 32-bit indices, unpacked settings, builtin algebra, shared library, no MKL or CUDA.
• Required build switches: OSQP_USE_FLOAT=OFF, OSQP_USE_LONG=OFF, OSQP_PACK_SETTINGS=OFF, OSQP_ALGEBRA_BACKEND=builtin, OSQP_BUILD_SHARED_LIB=ON.
• Native library filename in source and deployed plugin is exactly osqp.dll.
• ClumsyPilot.dll locates and preloads the sibling DLL from Assembly.Location; current directory and system PATH are not inputs.
• Every native entry point uses CallingConvention.Cdecl.
• All pinned arrays, CSC wrappers, settings, and solver handles are released in reverse acquisition order.
• SolvedInaccurate is publishable only after independent strict residual and domain validation.
• Missing DLL, wrong architecture, version mismatch, invalid exports, and setup failure return structured solver outcomes.
• Native verbose output is disabled.

---

Locked File Structure

ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/
├── IQpSolver.cs
├── QpSolveResult.cs
├── QpSolveStatus.cs
├── QpSolverSettings.cs
├── QuadraticProgram.cs
├── SparseCscMatrix.cs
├── SparseTripletBuilder.cs
└── Osqp/
    ├── OsqpNativeLoader.cs
    ├── OsqpNativeMethods.cs
    ├── OsqpNativeSolver.cs
    ├── OsqpNativeStructures.cs
    └── OsqpStatusMapper.cs

ClumsyPilot/ThirdParty/OSQP/
├── build-win-x64.ps1
├── LICENSE
├── NOTICE
├── VERSION
├── SHA256SUMS
└── win-x64/osqp.dll

ClumsyPilot/tests/EMPlannerVerificationHost/
├── OptimizationChecks.cs
└── OsqpChecks.cs

Shared Interfaces

public interface IQpSolver
{
    QpSolveResult Solve(QuadraticProgram problem, QpSolverSettings settings,
        IReadOnlyList<double> warmStart, CancellationToken cancellationToken);
}

public sealed class QuadraticProgram
{
    public QuadraticProgram(SparseCscMatrix upperTriangularP, IReadOnlyList<double> q,
        SparseCscMatrix a, IReadOnlyList<double> lowerBounds,
        IReadOnlyList<double> upperBounds);
    public int VariableCount { get; }
    public int ConstraintCount { get; }
}

public sealed class QpSolveResult
{
    public QpSolveStatus Status { get; }
    public IReadOnlyList<double> Primal { get; }
    public double Objective { get; }
    public double PrimalResidual { get; }
    public double DualResidual { get; }
    public int Iterations { get; }
    public TimeSpan SolveTime { get; }
    public string NativeStatus { get; }
    public string Diagnostic { get; }
}

Official references used to lock this ABI:

• https://osqp.org/docs/interfaces/C.html
• https://osqp.org/docs/get_started/migration_guide.html
• https://github.com/osqp/osqp/tree/v1.0.0

Task 1: Solver-Neutral Sparse QP Contracts

Files:

• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/IQpSolver.cs
• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/QpSolveResult.cs
• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/QpSolveStatus.cs
• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/QpSolverSettings.cs
• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/QuadraticProgram.cs
• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/SparseCscMatrix.cs
• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/SparseTripletBuilder.cs
• Create: ClumsyPilot/tests/EMPlannerVerificationHost/OptimizationChecks.cs
• Modify: ClumsyPilot/tests/EMPlannerVerificationHost/Program.cs

Interfaces:

• Consumes: NumericGuard and cancellation tokens.

• Produces: the shared interfaces above and deterministic sparse-matrix assembly used by LS and ST.

• Step 1: Write failing CSC canonicalization checks

Build triplets in shuffled order with duplicate coordinates and assert the resulting CSC matrix:

has ColumnPointers length ColumnCount+1
sorts row indices ascending inside each column
sums duplicate coordinates
drops exact zero sums
rejects NaN, infinity, negative indices, and out-of-range indices
stores only the upper triangle for P

Also construct the micro problem min 0.5*x^2 - 2*x subject to 0 <= x <= 1 and assert its immutable arrays cannot be changed through the source lists.

• Step 2: Run and verify solver contracts are absent

dotnet run --project ClumsyPilot/tests/EMPlannerVerificationHost/EMPlannerVerificationHost.csproj -- optimization

Expected: build failure naming SparseTripletBuilder.

• Step 3: Implement canonical CSC and QP validation

SparseCscMatrix stores copied arrays Values, RowIndices, and ColumnPointers. Validate monotonic pointers, ColumnPointers[0]==0, final pointer equals nonzero count, and all rows are in range. QuadraticProgram enforces square P, matching variable dimensions, matching constraint dimensions, lower<=upper, finite coefficients, and bounds limited to ±1e30 rather than CLR infinity.

Use these exact statuses:

public enum QpSolveStatus
{
    Solved,
    SolvedInaccurate,
    PrimalInfeasible,
    DualInfeasible,
    MaximumIterations,
    TimeLimit,
    Cancelled,
    SolverUnavailable,
    InvalidProblem,
    NativeError
}

• Step 4: Run optimization checks

Expected: PASS optimization.

• Step 5: Commit QP contracts

git add ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization ClumsyPilot/tests/EMPlannerVerificationHost
git commit -m "feat: add solver-neutral QP contracts"

Task 2: Reproducible OSQP 1.0.0 Native Package

Files:

• Create: all files under ClumsyPilot/ThirdParty/OSQP/

Interfaces:

• Consumes: Git, CMake 3.18+, and a Visual Studio x64 compiler.

• Produces: a versioned win-x64/osqp.dll with a recorded SHA-256 and matching license files.

• Step 1: Write the native build script

build-win-x64.ps1 must create a unique temporary directory, clone only tag v1.0.0, configure with this exact command shape, and remove the temporary directory in finally:

cmake -S $sourceRoot -B $buildRoot -A x64 `
  -DOSQP_ALGEBRA_BACKEND=builtin `
  -DOSQP_BUILD_SHARED_LIB=ON `
  -DOSQP_BUILD_STATIC_LIB=OFF `
  -DOSQP_BUILD_DEMO_EXE=OFF `
  -DOSQP_BUILD_UNITTESTS=OFF `
  -DOSQP_USE_FLOAT=OFF `
  -DOSQP_USE_LONG=OFF `
  -DOSQP_PACK_SETTINGS=OFF `
  -DOSQP_ENABLE_PRINTING=OFF `
  -DOSQP_CODEGEN=OFF `
  -DOSQP_ENABLE_DERIVATIVES=OFF
cmake --build $buildRoot --config Release --target osqp

The script resolves the generated DLL explicitly, verifies exactly one match, copies upstream LICENSE and NOTICE, writes VERSION with tag and build flags, computes Get-FileHash -Algorithm SHA256, and writes SHA256SUMS using a lowercase hexadecimal digest.

• Step 2: Execute the build script

powershell -ExecutionPolicy Bypass -File ClumsyPilot/ThirdParty/OSQP/build-win-x64.ps1

Expected: win-x64/osqp.dll, LICENSE, NOTICE, VERSION, and SHA256SUMS exist; the script prints OSQP v1.0.0 win-x64 package ready.

• Step 3: Verify architecture, exports, and hash

Use dumpbin /headers to assert machine x64, dumpbin /exports to assert osqp_version, osqp_setup, osqp_solve, and osqp_cleanup, then recompute SHA-256 and compare with SHA256SUMS. A missing tool is a failed packaging gate, not a skipped check.

• Step 4: Verify license contents came from the pinned tag

Compare bytes against the tag checkout before the temporary checkout is removed. Expected: exact equality for both files.

• Step 5: Commit the reproducible native package

git add ClumsyPilot/ThirdParty/OSQP
git commit -m "build: pin OSQP 1.0.0 win-x64"

Task 3: Absolute-Path Native Loader and ABI Structures

Files:

• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpNativeLoader.cs
• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpNativeMethods.cs
• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpNativeStructures.cs
• Create: ClumsyPilot/tests/EMPlannerVerificationHost/OsqpChecks.cs
• Modify: ClumsyPilot/tests/EMPlannerVerificationHost/Program.cs

Interfaces:

• Consumes: the pinned DLL and Task 1 QP contracts.

• Produces: a process-lifetime loader plus exact OSQP 1.0.0 double/int32 structures.

• Step 1: Write failing loader checks

Copy the verification host output to a temporary plugins directory with ClumsyPilot.dll and, in separate cases, no native DLL, a text file named osqp.dll, and the real DLL. Assert the first two return SolverUnavailable diagnostics without BadImageFormatException escaping; the real DLL reports version 1.0.0. Start 16 parallel first-use calls and assert a single stable module handle.

• Step 2: Run the OSQP group without loader implementation

Expected: build failure naming OsqpNativeLoader.

• Step 3: Implement loader and ABI definitions

Use Windows LoadLibraryW, GetProcAddress, and FreeLibrary from kernel32; resolve the plugin directory from typeof(OsqpNativeLoader).Assembly.Location. Reject IntPtr.Size != 8. Preload the absolute sibling path and retain the handle for process lifetime.

Define OSQPInt as C# int and OSQPFloat as C# double, matching the pinned build. Define sequential layouts for OSQPCscMatrix, OSQPSettings, OSQPInfo, OSQPSolution, and the four-pointer prefix of OSQPSolver exactly as the v1.0.0 public headers specify. Add an internal layout check for expected offsets and total sizes before the first solve.

Declare only these native functions initially:

osqp_version
osqp_set_default_settings
osqp_setup
osqp_warm_start
osqp_solve
osqp_cleanup

Do not depend on OSQPCscMatrix_new, OSQPCscMatrix_free, OSQPSettings_new, or OSQPSettings_free: those helpers are not marked with the public export macro in the pinned header. Allocate the two CSC structures and settings block with Marshal.AllocHGlobal, initialize settings through osqp_set_default_settings, and release those managed-owned blocks with Marshal.FreeHGlobal.

• Step 4: Run missing, corrupt, real, and concurrent loader checks

Expected: PASS osqp-loader and no process crash.

• Step 5: Commit loader and structures

git add ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp ClumsyPilot/tests/EMPlannerVerificationHost
git commit -m "feat: load pinned OSQP native library"

Task 4: OSQP Solve Lifecycle and Status Mapping

Files:

• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpNativeSolver.cs
• Create: ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp/OsqpStatusMapper.cs
• Modify: ClumsyPilot/tests/EMPlannerVerificationHost/OsqpChecks.cs

Interfaces:

• Consumes: IQpSolver, Task 3 native functions, and QpSolverSettings.

• Produces: OsqpNativeSolver : IQpSolver.

• Step 1: Write failing solve/status checks

Test three fixed QPs:

bounded optimum: min 0.5*x^2 - 2*x, 0<=x<=1, expected x=1
equality optimum: min x^2+y^2, x+y=1, expected x=y=0.5
infeasible: x>=1 and x<=0, expected PrimalInfeasible

Assert residuals, iteration count, objective, native status, and solve time are populated. Add a 1e-9 second time-limit case that maps only to TimeLimit or a valid solved status; no native status may be silently treated as solved.

• Step 2: Run and verify OsqpNativeSolver is absent

Expected: build failure naming OsqpNativeSolver.

• Step 3: Implement one-shot native ownership

Pin P/Q/A/L/U and optional warm-start arrays; allocate and populate P/A OSQPCscMatrix blocks; allocate settings and initialize it through osqp_set_default_settings; overwrite verbose=0, warm_starting, polishing, max_iter, eps_abs, eps_rel, and time_limit; call setup, optional warm start, solve, then marshal solution and info. Copy all result values before cleanup. Release the solver through osqp_cleanup, then settings/matrix blocks through Marshal.FreeHGlobal, then array pins in reverse order inside finally.

Map native status values exactly:

1 Solved
2 SolvedInaccurate
3/4 PrimalInfeasible
5/6 DualInfeasible
7 MaximumIterations
8 TimeLimit
9/10/11 NativeError

Cancellation is checked before native setup and after solve. OSQP's configured time limit is the bound for a solve already inside native code.

• Step 4: Run all OSQP checks repeatedly

1..20 | ForEach-Object {
  dotnet run --project ClumsyPilot/tests/EMPlannerVerificationHost/EMPlannerVerificationHost.csproj -- osqp
  if ($LASTEXITCODE -ne 0) { throw "OSQP verification failed on iteration $_" }
}

Expected: every iteration prints PASS osqp-loader and PASS osqp-solve.

• Step 5: Commit solver lifecycle

git add ClumsyPilot/ParkrobTrajplanner/EMPlanner/Optimization/Osqp ClumsyPilot/tests/EMPlannerVerificationHost/OsqpChecks.cs
git commit -m "feat: solve QPs through OSQP"

Task 5: Backend Completion Gate

Files:

• Modify: ClumsyPilot/ParkrobTrajplanner/EMPlanner/README.md
• Modify: ClumsyPilot/tests/EMPlannerVerificationHost/Program.cs

Interfaces:

• Consumes: all tasks in this plan.

• Produces: the stable IQpSolver boundary required by LS and ST plans.

• Step 1: Document native deployment and diagnostics

Add the exact source/deployment layouts, pinned version, build flags, license placement, absolute loading rule, and solver status mapping to the README.

• Step 2: Run optimization and OSQP gates

dotnet run --project ClumsyPilot/tests/EMPlannerVerificationHost/EMPlannerVerificationHost.csproj -- optimization
dotnet run --project ClumsyPilot/tests/EMPlannerVerificationHost/EMPlannerVerificationHost.csproj -- osqp
git diff --check

Expected: both groups pass and Git reports no whitespace errors.

• Step 3: Verify the DLL is self-contained

Run a dependency inspection on win-x64/osqp.dll. Expected: only Windows system/runtime DLLs; no MKL, CUDA, or separately deployed QDLDL DLL.

• Step 4: Verify clean plugin-directory loading

Copy only ClumsyPilot.dll and osqp.dll to a fresh directory, copy the verification host executable beside them, and run the micro QP with the working directory set elsewhere. Expected: solved result, proving loading does not depend on current directory.

• Step 5: Commit backend documentation

git add ClumsyPilot/ParkrobTrajplanner/EMPlanner/README.md ClumsyPilot/tests/EMPlannerVerificationHost/Program.cs
git commit -m "docs: describe OSQP plugin deployment"

Completion Gate

• IQpSolver contains no OSQP-specific type.
• Pinned native metadata, license, notice, hash, and DLL agree with OSQP v1.0.0.
• Loader failures are structured and never terminate the host.
• Fixed feasible and infeasible QPs map to the correct statuses with finite diagnostics.
• Twenty repeated solve/cleanup cycles pass without handle growth or access violations.