chore(task): archive 08-13-native-road-compiler
This commit is contained in:
@@ -0,0 +1,3 @@
|
||||
{"file":".trellis/spec/pipeline/index.md","reason":"Check command ownership, disk artifact boundaries, and legacy pipeline compatibility."}
|
||||
{"file":".trellis/spec/preview/index.md","reason":"Check browser workbench state, error handling, and no-build browser constraints."}
|
||||
{"file":".trellis/spec/config/index.md","reason":"Check new area output/config normalization and compatibility."}
|
||||
@@ -0,0 +1,110 @@
|
||||
# Native Road Compiler Workbench Design
|
||||
|
||||
## Architecture
|
||||
|
||||
The native compiler is an additive pipeline path. It owns a separate output
|
||||
directory under an area and never writes into `osm2streets_web_out/`.
|
||||
|
||||
```text
|
||||
OSM XML + native-road-overrides.json
|
||||
|
|
||||
v
|
||||
Canonical Road Model
|
||||
roads / endpoints / junction candidates / provenance
|
||||
|
|
||||
v
|
||||
Native Geometry Compiler
|
||||
road surfaces / initial junction surfaces / diagnostics
|
||||
|
|
||||
+--> native-road/compiled.json
|
||||
+--> native-road/layers/*.geojson
|
||||
+--> native-road/diagnostics.json
|
||||
+--> native-road/comparison.json
|
||||
|
|
||||
v
|
||||
Road Workbench HTTP service
|
||||
browser map + inspect/edit/save API
|
||||
```
|
||||
|
||||
The canonical model is the authority. Render layers, browser display data, and
|
||||
future Blender compatibility adapters are derived from it.
|
||||
|
||||
## Commands And Ownership
|
||||
|
||||
- `npm run road:compile -- --config <area-config>` performs no browser work.
|
||||
It reads OSM plus the persisted override file, writes a staged native-road
|
||||
result, validates it, and atomically promotes the result directory.
|
||||
- `npm run road:workbench -- --config <area-config>` compiles first unless
|
||||
`--no-compile` is supplied, then starts a local HTTP server scoped to that
|
||||
one area.
|
||||
- The server exposes read-only compiler artifacts and one explicit save API
|
||||
for validated overrides. It does not expose arbitrary filesystem paths.
|
||||
- Existing `build:area`, `intermediates`, QGIS, Blender, Cesium, and package
|
||||
paths remain unchanged in the first iteration.
|
||||
|
||||
## Data Contracts
|
||||
|
||||
### Canonical road model
|
||||
|
||||
Each road direction carries a stable ID derived from OSM identifiers, source
|
||||
way IDs, endpoint node IDs, centerline, explicit/inferred attributes, applied
|
||||
override IDs, and diagnostics. Junction candidates likewise use their OSM node
|
||||
ID when available. Values include provenance such as `tag:lanes:forward`,
|
||||
`inferred:highway-default`, or `override:<id>`.
|
||||
|
||||
Each `Movement` is a stable semantic record joining a connection, source and
|
||||
target road/lane, turn class, provenance, and an optional connector geometry.
|
||||
`geometryStatus="connector"` publishes a connector curve;
|
||||
`geometryStatus="continuous"` means the lane centerlines meet at the node and
|
||||
does not invent a zero-length curve; `deferred-too-long` retains the movement
|
||||
while withholding unsafe geometry.
|
||||
|
||||
### Override file
|
||||
|
||||
`<area>/native-road-overrides.json` is versioned and human-reviewable. It
|
||||
contains an array of uniquely identified changes whose targets are stable road
|
||||
or endpoint IDs. Supported v1 records are `road` parameter overrides and
|
||||
`junction-connection` decisions. The save endpoint validates schema, target
|
||||
existence, finite values, and duplicate/conflicting edits before atomic write.
|
||||
|
||||
### Compiler artifacts
|
||||
|
||||
`<area>/native-road/compiled.json` is the workbench's single read model and
|
||||
contains the canonical `movements` list.
|
||||
`layers/` contains generated GeoJSON with source/provenance properties.
|
||||
`diagnostics.json` contains severity, stable subject ID, source IDs, rule,
|
||||
message, and optional geometry. `comparison.json` reports counts and coverage
|
||||
against available osm2streets layers; it does not claim quality solely from
|
||||
visual differences.
|
||||
|
||||
## Browser Workbench
|
||||
|
||||
The browser uses OpenLayers as its sole GIS runtime, served directly from the
|
||||
local allowlisted `node_modules` packages with a browser import map. The map
|
||||
renders fit-to-data OSM centerlines, native surfaces, optional osm2streets
|
||||
reference layers, diagnostics, selected-object provenance, movements, and
|
||||
overrides. This provides mature map selection and hit detection without a
|
||||
framework or bundler.
|
||||
|
||||
The user can select a road or endpoint, edit only v1 fields, inspect the
|
||||
resulting override record, explicitly save it, and recompile/reload. Saved
|
||||
state is visibly differentiated from unsaved state. The workbench must not
|
||||
offer freehand final-polygon editing, since that would break reproducibility.
|
||||
|
||||
## Geometry And Validation
|
||||
|
||||
V1 produces road segments from projected centerline offsets and terminal
|
||||
cross-sections. It only generates a junction surface when endpoints satisfy
|
||||
the supported ordinary T/cross shape and geometry checks; otherwise it emits a
|
||||
diagnostic rather than inventing an invalid polygon. Validation detects
|
||||
dangling endpoints, unclosed/self-intersecting rings, non-finite coordinates,
|
||||
unsupported multi-level intersections, and source/topology ambiguity. Small
|
||||
numerical cleanup may be explicit and recorded; semantic failures are never
|
||||
silently repaired.
|
||||
|
||||
## Compatibility And Rollout
|
||||
|
||||
The first compiler's layers use existing render-layer names where meaningful,
|
||||
but are stored separately. A later, explicitly enabled Blender provider option
|
||||
may consume native layers after comparison gates pass. Delete/replace behavior
|
||||
is out of scope; rollback is selecting the existing osm2streets pipeline.
|
||||
@@ -0,0 +1,3 @@
|
||||
{"file":".trellis/spec/pipeline/index.md","reason":"Native compiler commands, artifacts, and area config extend the Node pipeline while preserving legacy stages."}
|
||||
{"file":".trellis/spec/preview/index.md","reason":"The browser workbench is a new DOM runtime and must follow local preview loading and state conventions where applicable."}
|
||||
{"file":".trellis/spec/config/index.md","reason":"New native-road output paths and config behavior extend the normalized area contract."}
|
||||
@@ -0,0 +1,32 @@
|
||||
# Implementation Plan
|
||||
|
||||
1. Add area output/config normalization and command entrypoints for the native
|
||||
compiler, preserving existing stage behavior and paths.
|
||||
2. Implement a shared OSM road parser and canonical road/endpoints model with
|
||||
source provenance, explicit versus inferred properties, and stable IDs.
|
||||
3. Implement v1 override schema, validation, load/apply behavior, atomic save,
|
||||
and focused unit tests.
|
||||
4. Implement projected road segment geometry, supported T/cross junction
|
||||
detection, native GeoJSON artifact emission, diagnostics, and comparison
|
||||
summary.
|
||||
5. Implement a dependency-free local workbench server and browser UI with
|
||||
selection, provenance display, v1 parameter/topology editing, explicit save,
|
||||
compile/reload, and error states.
|
||||
6. Add native compiler tests using focused fixtures plus nantaizi analysis;
|
||||
run existing relevant Node tests to confirm legacy behavior remains intact.
|
||||
7. Compare nantaizi and at least one supplied problematic OSM sample. Record
|
||||
metrics, unsupported cases, and follow-up work in task research.
|
||||
|
||||
## Validation
|
||||
|
||||
```bash
|
||||
npm run road:compile -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
npm run road:workbench -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
npm run test:build-stages
|
||||
npm run test:preflight
|
||||
npm run test:preview-assets
|
||||
```
|
||||
|
||||
Browser validation includes loading the workbench, editing a road parameter,
|
||||
saving, verifying the override file, recompiling, reloading, and confirming
|
||||
provenance identifies the saved override.
|
||||
@@ -0,0 +1,93 @@
|
||||
# Native road compiler workbench
|
||||
|
||||
## Goal
|
||||
|
||||
Build an incremental native road compiler for Chinese urban and campus OSM
|
||||
data that can progressively exceed osm2streets in geometry quality,
|
||||
explainability, and repeatable correction. The existing osm2streets pipeline
|
||||
must remain usable while the native compiler is developed and compared.
|
||||
|
||||
The first deliverable is a browser-based Road Workbench. It must expose the
|
||||
native compiler's source data, generated geometry, and diagnostics, allow
|
||||
users to make small semantic/topology corrections, persist those corrections
|
||||
as versionable overrides, and reload them automatically in later runs.
|
||||
|
||||
## Confirmed Facts
|
||||
|
||||
- Nantaizi currently works relatively well because its OSM data received
|
||||
deliberate supplemental tagging; it still has missing boundaries and
|
||||
polygons that cannot be closed.
|
||||
- Other tested OSM inputs expose osm2streets sensitivity to input structure
|
||||
and leave too much opaque, final-polygon repair work in QGIS.
|
||||
- Existing Blender consumes the nine GeoJSON render layers from
|
||||
`osm2streets_web_out/`; QGIS GeoPackage edits can currently be reimported
|
||||
only as a whole batch.
|
||||
- The repository has no existing interactive browser editing service. Existing
|
||||
Cesium preview is a static, generated verification page.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1: Add a native-road-compiler path without replacing or regressing the
|
||||
existing osm2streets path.
|
||||
- R2: Parse OSM into a canonical, source-traceable road model with stable
|
||||
references to OSM ways and nodes, explicit values versus inferred values,
|
||||
and diagnostics.
|
||||
- R3: Compile at least ordinary road segments and the initial supported
|
||||
junction subset into the existing render-layer contract, allowing existing
|
||||
Blender/Cesium consumers to be reused.
|
||||
- R4: Provide a browser Road Workbench that overlays raw OSM topology,
|
||||
generated geometry, osm2streets comparison geometry when available, and
|
||||
compiler diagnostics.
|
||||
- R5: The workbench must permit scoped user adjustments and save them to an
|
||||
area-local, human-reviewable override file. Future compile and workbench
|
||||
runs must load that file automatically.
|
||||
- R6: Each generated object and diagnostic must be traceable to OSM source
|
||||
IDs, compiler rule/inference evidence, and relevant override IDs.
|
||||
- R7: Validate topology and geometry before publishing generated layers;
|
||||
report unresolved semantic errors instead of silently disguising them as
|
||||
geometric repair.
|
||||
- R8: Develop against nantaizi plus problem inputs and report native versus
|
||||
osm2streets comparison metrics.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- First implementation targets Chinese urban/campus roads, ordinary road
|
||||
segments, T/cross junctions, directed/multi-lane roads, and data already
|
||||
tagged in nantaizi where possible.
|
||||
- Existing Blender, Cesium export, package format, building, vegetation, and
|
||||
water generators are out of scope unless a compatibility adapter requires a
|
||||
narrowly scoped change.
|
||||
- Directly editing final render polygons is not the intended correction model;
|
||||
generated layers remain derived output.
|
||||
- Complex interchanges, arbitrary multilayer junctions, and full worldwide OSM
|
||||
coverage are deferred until driven by concrete samples.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] A native compile command produces a canonical road model, generated
|
||||
layers, diagnostics, and comparison artifacts for a configured area without
|
||||
changing the osm2streets output path.
|
||||
- [ ] A browser command serves a Road Workbench for an area and clearly shows
|
||||
source topology, generated output, diagnostics, provenance, and saved
|
||||
overrides.
|
||||
- [ ] A user can make the agreed first-scope override edits in the browser,
|
||||
save them explicitly, and receive a durable area-local override artifact.
|
||||
- [ ] Re-running compile or reopening the workbench applies saved overrides
|
||||
automatically and exposes their provenance.
|
||||
- [ ] The compiler reports invalid/unclosed geometry, dangling road ends,
|
||||
and unresolved junction/lane ambiguity with source IDs.
|
||||
- [ ] Nantaizi and at least one known problematic area can run through the
|
||||
native analysis/preview path, with comparison metrics captured rather than
|
||||
a claim based only on visual inspection.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- The first browser editing surface supports road parameters (width, directed
|
||||
lane counts, left/right sidewalk state) plus junction endpoint
|
||||
connect/disconnect decisions.
|
||||
- Turn restrictions, stop lines, and crosswalk placement are deferred until
|
||||
the compiler has a validated road/junction editing loop.
|
||||
- Overrides are a versioned, human-reviewable JSON artifact owned by the area,
|
||||
not edits to generated polygon layers.
|
||||
- Native output and osm2streets output remain parallel during development;
|
||||
neither silently overwrites the other.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Two-Area Native Road Comparison
|
||||
|
||||
## Runs
|
||||
|
||||
2026-08-14:
|
||||
|
||||
```bash
|
||||
npm run road:compile -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
npm run build:area -- --config config/areas/hanyang-block.json --stages intermediates
|
||||
npm run road:compile -- --config config/areas/hanyang-block.json
|
||||
```
|
||||
|
||||
`comparison.json` is a coverage and diagnostic record. Feature counts are not
|
||||
a geometry-quality score: osm2streets and the native compiler segment roads at
|
||||
different levels.
|
||||
|
||||
| Area | Directional roads | Native surfaces | Native junctions | Movements | Internal ends | Manual candidates | osm2streets road surfaces |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| nantaizi-lake-innovation-valley | 34 | 19 | 6 | 46 | 4 | 2 | 50 |
|
||||
| hanyang-block | 475 | 352 | 20 | 369 | 161 | 83 | 1826 |
|
||||
|
||||
## Observed Failure Modes
|
||||
|
||||
The hanyang osm2streets/QGIS run completed, but its log reported repeated:
|
||||
|
||||
- roads trimmed into oblivion;
|
||||
- degenerate intersections that could not be collapsed because layers, names,
|
||||
highway types, or lane specifications differ;
|
||||
- intersection polygon requests with no roads.
|
||||
|
||||
The native compiler did not hide the related uncertainty. It reported 161
|
||||
internal road ends. Eighty-three have one or more nearby, direction-compatible
|
||||
candidate departures within 35 metres; these remain suggestions for explicit
|
||||
review rather than automatic topology edits. The initial five one-way junctions
|
||||
that lacked connector curves now publish road-surface envelopes based on their
|
||||
semantic movements. Across hanyang, 369 movements are identified: 243 require
|
||||
a connector curve, while 126 are continuous at their OSM node and intentionally
|
||||
have no separate geometry.
|
||||
|
||||
## Resulting Priorities
|
||||
|
||||
1. Keep manual candidate suggestions and semantic overrides as the correction
|
||||
path for near-miss topology. Do not bulk-connect candidates.
|
||||
2. Expand ordinary junction support from 3/4 physical approaches only after
|
||||
identifying a repeated unsupported topology; one-way movements that are
|
||||
continuous at a node are already supported without fake connector geometry.
|
||||
3. Add an inspectable movement artifact so turn geometry is not the only
|
||||
representation of a road-to-road movement.
|
||||
4. Use a visual review of a few explicit hanyang diagnostics before changing
|
||||
connection-distance or road-class rules.
|
||||
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"id": "native-road-compiler",
|
||||
"name": "native-road-compiler",
|
||||
"title": "Native road compiler workbench",
|
||||
"description": "",
|
||||
"status": "completed",
|
||||
"dev_type": null,
|
||||
"scope": null,
|
||||
"package": null,
|
||||
"priority": "P2",
|
||||
"creator": "dingkang",
|
||||
"assignee": "dingkang",
|
||||
"createdAt": "2026-08-13",
|
||||
"completedAt": "2026-08-18",
|
||||
"branch": null,
|
||||
"base_branch": "feature/native-road-compiler",
|
||||
"worktree_path": null,
|
||||
"commit": null,
|
||||
"pr_url": null,
|
||||
"subtasks": [],
|
||||
"children": [
|
||||
"08-14-native-road-lane-markings",
|
||||
"08-17-native-road-control-markings",
|
||||
"08-17-native-road-center-lines",
|
||||
"08-17-native-rounded-junctions",
|
||||
"08-18-native-traffic-signal-parity"
|
||||
],
|
||||
"parent": null,
|
||||
"relatedFiles": [],
|
||||
"notes": "",
|
||||
"meta": {}
|
||||
}
|
||||
Reference in New Issue
Block a user