feat: add reimport stage for hand-edited GeoPackages
QGIS 手工修正后的回导流程此前是 README 里的三段 shell:一个硬编码
area-id 和 ogr2ogr 绝对路径的 for 循环、一段内联 node heredoc、再加一次
npm run build。改为一个 reimport 阶段:
npm run build -- --config config/areas/<area-id>.json \
--stages reimport,blender,cesium
scripts/reimport-gpkg.js 先把全部图层导出到临时目录并逐个校验,全部通过
才写回 osm2streets_web_out/。ogr2ogr 对不存在的图层退出码非 0 但仍会留下
0 字节文件,原先逐图层 mv 会静默用空图层覆盖好数据。
intermediates 与 reimport 同时指定直接报错——前者用 OSM 重建 GeoPackage,
正好抹掉后者要读回的手工修改。reimport 不含在 all 中。
同时新增 scripts/lib/scene-layers.js 作为 9 个渲染图层的唯一定义源。此前
该表在合并场景、场景样式 JSON、生成的 QGIS 工程、README 手工流程中各有
一份副本,改一处漏其余会导致图层叠放顺序出错并流入 Blender/Cesium。
验证:同一 OSM 输入下,改动前后 intermediates 产物逐字节一致
(scene_style.json、.qgz 符号定义、9 个图层几何与属性);reimport 对
未修改图层无损回导。
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
31
README.md
31
README.md
@@ -50,9 +50,10 @@ npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages blender
|
||||
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages cesium
|
||||
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages preview
|
||||
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages reimport
|
||||
```
|
||||
|
||||
`intermediates` 会生成 osm2streets GeoJSON、GeoPackage、QGIS 工程和 QGIS 预览图。`blender` 使用 OSM 和 osm2streets GeoJSON 生成 `.blend`/`.png`。`cesium` 从 `.blend` 导出 `.glb`/`.json`,并生成 Cesium 预览 HTML。`preview` 只在已有 `.glb/.json` 时补生成 HTML。
|
||||
`intermediates` 会生成 osm2streets GeoJSON、GeoPackage、QGIS 工程和 QGIS 预览图。`blender` 使用 OSM 和 osm2streets GeoJSON 生成 `.blend`/`.png`。`cesium` 从 `.blend` 导出 `.glb`/`.json`,并生成 Cesium 预览 HTML。`preview` 只在已有 `.glb/.json` 时补生成 HTML。`reimport` 把手工编辑过的 GeoPackage 回导为 GeoJSON,不含在 `all` 里,详见 [QGIS 手工修正工作流](#qgis-手工修正工作流)。
|
||||
|
||||
## 区域配置
|
||||
|
||||
@@ -169,6 +170,34 @@ Cesium GLB 低层导出:
|
||||
|
||||
详见 [blender/README.md](blender/README.md)。
|
||||
|
||||
## QGIS 手工修正工作流
|
||||
|
||||
如果已经在现成的 `.qgz` 项目里直接编辑了 `gpkg` 图层,不要再重跑 `intermediates`,否则会把手工修改覆盖掉。推荐流程是:
|
||||
|
||||
1. 在 QGIS 中打开 `outputs/<area-id>/<area-id>.qgz`
|
||||
2. 直接编辑项目内关联的 `gpkg` 图层并保存
|
||||
3. 跑 `reimport` 回导 GeoJSON 并重建场景,再接 `blender,cesium`
|
||||
|
||||
南台子湖创新谷当前可直接使用下面这条命令:
|
||||
|
||||
```bash
|
||||
npm run build -- --config config/areas/nantaizi-lake-innovation-valley.json --stages reimport,blender,cesium
|
||||
```
|
||||
|
||||
`reimport` 阶段(`scripts/reimport-gpkg.js`)做两件事:
|
||||
|
||||
- 把 `<area-id>.gpkg` 里的 9 个图层逐个导出到 `osm2streets_web_out/<layer>.geojson`
|
||||
- 按 `scripts/lib/scene-layers.js` 的图层表重建 `osm2streets_scene.geojson`(写入 `render_layer` / `z_index`)和 `osm2streets_scene_style.json`
|
||||
|
||||
说明:
|
||||
|
||||
- 这套流程假设你的手工修改已经保存在 `outputs/<area-id>/<area-id>.gpkg` 中
|
||||
- 所有图层先导出到临时目录并校验通过后才写回 `osm2streets_web_out/`;任一图层缺失或导出结果不是合法 FeatureCollection,整批都不落盘(`ogr2ogr` 遇到不存在的图层会留下 0 字节文件,直接覆盖会静默损坏数据)
|
||||
- `intermediates` 与 `reimport` 互斥,同时指定会直接报错:前者用 OSM 重建 `gpkg`,正好会抹掉后者要读回的手工修改
|
||||
- `blender,cesium` 阶段读取的是 `osm2streets_web_out/*.geojson`,不是直接读取 `gpkg`
|
||||
- 如果 Blender 当前环境不稳定,先确认 `geojson` 已完成回导,再单独排查 Blender 本身
|
||||
- 增删图层或调整 `z_index` 只需改 `scripts/lib/scene-layers.js`,构建、场景合并、场景样式、QGIS 工程会一并同步
|
||||
|
||||
## 文档
|
||||
|
||||
- [docs/input-cases.md](docs/input-cases.md) - 已验证的 OSM 输入案例
|
||||
|
||||
@@ -1,5 +1,15 @@
|
||||
# Changelog
|
||||
|
||||
## 2026-07-28
|
||||
|
||||
- 新增 `reimport` 阶段(`scripts/reimport-gpkg.js`),把 QGIS 手工修正过的 `<area-id>.gpkg` 回导为 `osm2streets_web_out/*.geojson` 并重建 `osm2streets_scene.geojson` / `osm2streets_scene_style.json`。手工修正流程从「`ogr2ogr` 循环 + 内联 node 脚本 + `npm run build`」三步压缩为一条命令:
|
||||
```bash
|
||||
npm run build -- --config config/areas/<area-id>.json --stages reimport,blender,cesium
|
||||
```
|
||||
- `reimport` 先把全部图层导出到临时目录并逐个校验,全部通过才写回输出目录:`ogr2ogr` 对不存在的图层退出码非 0 但仍会留下 0 字节文件,逐图层直接覆盖会静默损坏数据。
|
||||
- `intermediates` 与 `reimport` 同时指定时直接报错:前者会用 OSM 重建 GeoPackage,正好抹掉后者要读回的手工修改。`reimport` 不含在 `all` 中。
|
||||
- 新增 `scripts/lib/scene-layers.js` 作为 9 个渲染图层的唯一定义源(id、`z_index`、配色、描边宽度)。此前该表在合并场景、场景样式 JSON、生成的 QGIS 工程、README 手工流程中各有一份副本,改一处漏其余会导致图层叠放顺序错误并流入 Blender/Cesium。构建产物(GeoJSON、GeoPackage、`.qgz` 符号、样式 JSON)与改动前逐字节一致。
|
||||
|
||||
## 2026-07-27
|
||||
|
||||
- 实验分支新增 Cesium 车辆巡航预览:从 OSM 可行驶 `highway` 提取 bounds 内路线,输出 `<area-id>-vehicle-route.json`,并在预览页中驱动车辆循环移动。
|
||||
|
||||
@@ -15,6 +15,16 @@ const requestedStages = args.stages
|
||||
: null;
|
||||
const stages = resolveStages(area.stages, requestedStages);
|
||||
|
||||
// intermediates deletes and rebuilds the GeoPackage from OSM, which is exactly
|
||||
// the manual work reimport exists to recover. Refuse the combination instead of
|
||||
// silently letting one undo the other.
|
||||
if (stages.intermediates && stages.reimport) {
|
||||
throw new Error(
|
||||
"Stages 'intermediates' and 'reimport' are mutually exclusive: " +
|
||||
"intermediates rebuilds the GeoPackage from OSM and would discard the QGIS edits reimport reads back.",
|
||||
);
|
||||
}
|
||||
|
||||
console.log(`Area: ${area.id}`);
|
||||
console.log(`Config: ${configPath}`);
|
||||
console.log(`Output: ${area.outputs.areaDir}`);
|
||||
@@ -22,6 +32,9 @@ console.log(`Output: ${area.outputs.areaDir}`);
|
||||
if (stages.intermediates) {
|
||||
buildIntermediates(area);
|
||||
}
|
||||
if (stages.reimport) {
|
||||
reimportGpkg(area);
|
||||
}
|
||||
if (stages.blender) {
|
||||
buildBlenderScene(area);
|
||||
}
|
||||
@@ -97,6 +110,7 @@ function normalizeAreaConfig(raw) {
|
||||
intermediates: raw.stages?.intermediates ?? raw.stages?.qgis ?? true,
|
||||
blender: raw.stages?.blender ?? true,
|
||||
cesium: raw.stages?.cesium ?? true,
|
||||
reimport: false,
|
||||
preview: false,
|
||||
},
|
||||
qgis: {
|
||||
@@ -142,6 +156,8 @@ function splitList(value) {
|
||||
|
||||
function resolveStages(defaults, requested) {
|
||||
if (!requested) return defaults;
|
||||
// 'reimport' is deliberately absent from 'all': it is a recovery step for
|
||||
// hand-edited GeoPackages, never part of a full build.
|
||||
const aliases = {
|
||||
all: ["intermediates", "blender", "cesium"],
|
||||
qgis: ["intermediates"],
|
||||
@@ -149,6 +165,8 @@ function resolveStages(defaults, requested) {
|
||||
geojson: ["intermediates"],
|
||||
intermediate: ["intermediates"],
|
||||
intermediates: ["intermediates"],
|
||||
reimport: ["reimport"],
|
||||
gpkg: ["reimport"],
|
||||
blender: ["blender"],
|
||||
scene: ["blender"],
|
||||
cesium: ["cesium"],
|
||||
@@ -157,18 +175,18 @@ function resolveStages(defaults, requested) {
|
||||
html: ["preview"],
|
||||
cesiumPreview: ["preview"],
|
||||
};
|
||||
const out = { intermediates: false, blender: false, cesium: false, preview: false };
|
||||
const out = { intermediates: false, reimport: false, blender: false, cesium: false, preview: false };
|
||||
for (const stage of requested) {
|
||||
const mapped = aliases[stage];
|
||||
if (!mapped) {
|
||||
throw new Error(`Unknown stage '${stage}'. Use intermediates, blender, cesium, preview, or all.`);
|
||||
throw new Error(`Unknown stage '${stage}'. Use intermediates, reimport, blender, cesium, preview, or all.`);
|
||||
}
|
||||
for (const key of mapped) out[key] = true;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function buildIntermediates(area) {
|
||||
function writeDerivedConfig(area) {
|
||||
fs.mkdirSync(area.outputs.pipelineDir, { recursive: true });
|
||||
const derivedConfig = {
|
||||
qgisApp: area.qgisApp,
|
||||
@@ -191,6 +209,11 @@ function buildIntermediates(area) {
|
||||
};
|
||||
const derivedConfigPath = path.join(area.outputs.pipelineDir, "osm2streets-qgis.config.json");
|
||||
fs.writeFileSync(derivedConfigPath, `${JSON.stringify(derivedConfig, null, 2)}\n`);
|
||||
return derivedConfigPath;
|
||||
}
|
||||
|
||||
function buildIntermediates(area) {
|
||||
const derivedConfigPath = writeDerivedConfig(area);
|
||||
|
||||
console.log("Stage: intermediates (osm2streets GeoJSON + QGIS)");
|
||||
runCommand(process.execPath, [
|
||||
@@ -200,6 +223,17 @@ function buildIntermediates(area) {
|
||||
], "intermediates");
|
||||
}
|
||||
|
||||
function reimportGpkg(area) {
|
||||
const derivedConfigPath = writeDerivedConfig(area);
|
||||
|
||||
console.log("Stage: reimport (GeoPackage -> GeoJSON)");
|
||||
runCommand(process.execPath, [
|
||||
path.join(repoRoot, "scripts", "reimport-gpkg.js"),
|
||||
"--config",
|
||||
derivedConfigPath,
|
||||
], "reimport");
|
||||
}
|
||||
|
||||
function buildBlenderScene(area) {
|
||||
ensureFile(blenderExecutable(area), "Blender executable");
|
||||
ensureFile(path.join(repoRoot, "blender", "generate_scene.py"), "Blender scene generator");
|
||||
|
||||
@@ -5,6 +5,15 @@ const path = require("path");
|
||||
const os = require("os");
|
||||
const { execFileSync } = require("child_process");
|
||||
const { JsStreetNetwork } = require("osm2streets-js-node");
|
||||
const {
|
||||
SCENE_LAYERS,
|
||||
SCENE_FILE,
|
||||
SCENE_STYLE_FILE,
|
||||
layerFile,
|
||||
mergeScene,
|
||||
sceneStyle,
|
||||
qgisRgba,
|
||||
} = require("./lib/scene-layers");
|
||||
|
||||
const repoRoot = path.resolve(__dirname, "..");
|
||||
const args = parseArgs(process.argv.slice(2));
|
||||
@@ -79,22 +88,16 @@ const split = splitLayers(
|
||||
intersectionCornerSourceMaxDimensionMeters,
|
||||
osm,
|
||||
);
|
||||
writeJson(path.join(outDir, "road_surface.geojson"), split.roadSurface);
|
||||
writeJson(path.join(outDir, "intersection_surface.geojson"), split.intersectionSurface);
|
||||
writeJson(path.join(outDir, "sidewalks.geojson"), split.sidewalks);
|
||||
writeJson(path.join(outDir, "lane_separators.geojson"), split.laneSeparators);
|
||||
writeJson(path.join(outDir, "center_lines.geojson"), split.centerLines);
|
||||
writeJson(path.join(outDir, "vehicle_stop_lines.geojson"), split.vehicleStopLines);
|
||||
writeJson(path.join(outDir, "lane_arrows_webscale.geojson"), split.laneArrows);
|
||||
for (const layer of SCENE_LAYERS) {
|
||||
writeJson(path.join(outDir, layerFile(layer)), split[layer.splitKey]);
|
||||
}
|
||||
if (arrowMergeTriangles) {
|
||||
normalizeLaneArrows(path.join(outDir, "lane_arrows_webscale.geojson"), arrowOutlineSimplifyMeters);
|
||||
split.laneArrows = JSON.parse(fs.readFileSync(path.join(outDir, "lane_arrows_webscale.geojson"), "utf8"));
|
||||
}
|
||||
writeJson(path.join(outDir, "sidewalk_corners.geojson"), split.sidewalkCorners);
|
||||
writeJson(path.join(outDir, "crosswalks.geojson"), split.crosswalks);
|
||||
writeJson(path.join(outDir, "osm2streets_scene.geojson"), mergedScene(split));
|
||||
writeJson(path.join(outDir, SCENE_FILE), mergeScene((layer) => split[layer.splitKey]));
|
||||
fs.writeFileSync(
|
||||
path.join(outDir, "osm2streets_scene_style.json"),
|
||||
path.join(outDir, SCENE_STYLE_FILE),
|
||||
JSON.stringify(sceneStyle(), null, 2),
|
||||
);
|
||||
|
||||
@@ -102,15 +105,10 @@ if (fs.existsSync(gpkgPath)) {
|
||||
fs.unlinkSync(gpkgPath);
|
||||
}
|
||||
const ogrEnv = qgisEnv();
|
||||
importLayer(gpkgPath, path.join(outDir, "road_surface.geojson"), "road_surface", false, ogrEnv);
|
||||
importLayer(gpkgPath, path.join(outDir, "intersection_surface.geojson"), "intersection_surface", true, ogrEnv);
|
||||
importLayer(gpkgPath, path.join(outDir, "sidewalks.geojson"), "sidewalks", true, ogrEnv);
|
||||
importLayer(gpkgPath, path.join(outDir, "sidewalk_corners.geojson"), "sidewalk_corners", true, ogrEnv);
|
||||
importLayer(gpkgPath, path.join(outDir, "lane_separators.geojson"), "lane_separators", true, ogrEnv);
|
||||
importLayer(gpkgPath, path.join(outDir, "center_lines.geojson"), "center_lines", true, ogrEnv);
|
||||
importLayer(gpkgPath, path.join(outDir, "vehicle_stop_lines.geojson"), "vehicle_stop_lines", true, ogrEnv);
|
||||
importLayer(gpkgPath, path.join(outDir, "lane_arrows_webscale.geojson"), "lane_arrows_webscale", true, ogrEnv);
|
||||
importLayer(gpkgPath, path.join(outDir, "crosswalks.geojson"), "crosswalks", true, ogrEnv);
|
||||
// First layer creates the GeoPackage; the rest append into it.
|
||||
SCENE_LAYERS.forEach((layer, index) => {
|
||||
importLayer(gpkgPath, path.join(outDir, layerFile(layer)), layer.id, index > 0, ogrEnv);
|
||||
});
|
||||
|
||||
const qgisScript = path.join(outDir, "_create_qgis_project.py");
|
||||
const previewFeature = split.crosswalks.features[0] || split.laneArrows.features[0] || split.roadSurface.features[0];
|
||||
@@ -406,53 +404,6 @@ function emptyCollection() {
|
||||
return { type: "FeatureCollection", features: [] };
|
||||
}
|
||||
|
||||
function mergedScene(split) {
|
||||
const layers = [
|
||||
["road_surface", 10, split.roadSurface],
|
||||
["intersection_surface", 20, split.intersectionSurface],
|
||||
["sidewalks", 30, split.sidewalks],
|
||||
["sidewalk_corners", 40, split.sidewalkCorners],
|
||||
["lane_separators", 50, split.laneSeparators],
|
||||
["center_lines", 60, split.centerLines],
|
||||
["crosswalks", 70, split.crosswalks],
|
||||
["vehicle_stop_lines", 80, split.vehicleStopLines],
|
||||
["lane_arrows_webscale", 90, split.laneArrows],
|
||||
];
|
||||
return {
|
||||
type: "FeatureCollection",
|
||||
features: layers.flatMap(([renderLayer, zIndex, collection]) => (
|
||||
(collection.features || []).map((feature) => ({
|
||||
...feature,
|
||||
properties: {
|
||||
...(feature.properties || {}),
|
||||
render_layer: renderLayer,
|
||||
z_index: zIndex,
|
||||
},
|
||||
}))
|
||||
)),
|
||||
};
|
||||
}
|
||||
|
||||
function sceneStyle() {
|
||||
return {
|
||||
version: 1,
|
||||
geometry: "polygon",
|
||||
sortProperty: "z_index",
|
||||
layerProperty: "render_layer",
|
||||
layers: [
|
||||
{ id: "road_surface", zIndex: 10, fill: "#2b2b28", outline: "#1e1e1c", outlineWidth: 0.04 },
|
||||
{ id: "intersection_surface", zIndex: 20, fill: "#2b2b28", outline: "#1e1e1c", outlineWidth: 0.04 },
|
||||
{ id: "sidewalks", zIndex: 30, fill: "#bebeb6", outline: "#9c9c94", outlineWidth: 0.025 },
|
||||
{ id: "sidewalk_corners", zIndex: 40, fill: "#bebeb6", outline: "#9c9c94", outlineWidth: 0.025 },
|
||||
{ id: "lane_separators", zIndex: 50, fill: "#eeeee6", outline: null, outlineWidth: 0 },
|
||||
{ id: "center_lines", zIndex: 60, fill: "#f5be2a", outline: null, outlineWidth: 0 },
|
||||
{ id: "crosswalks", zIndex: 70, fill: "#fffff6", outline: null, outlineWidth: 0 },
|
||||
{ id: "vehicle_stop_lines", zIndex: 80, fill: "#fffff6", outline: null, outlineWidth: 0 },
|
||||
{ id: "lane_arrows_webscale", zIndex: 90, fill: "#fffff6", outline: "#2b2b28", outlineWidth: 0.015 },
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
function splitLayers(dir, arrowScaleValue, maxCornerDimensionMeters, osm) {
|
||||
const plain = JSON.parse(fs.readFileSync(path.join(dir, "plain.geojson"), "utf8"));
|
||||
const lanePolygons = JSON.parse(fs.readFileSync(path.join(dir, "lane_polygons.geojson"), "utf8"));
|
||||
@@ -1360,6 +1311,16 @@ function importLayer(gpkg, source, layerName, update, env) {
|
||||
execFileSync(ogr2ogr, args, { stdio: "inherit", env: { ...process.env, ...env } });
|
||||
}
|
||||
|
||||
function qgisLayerSpecs() {
|
||||
return SCENE_LAYERS.map((layer) => ({
|
||||
id: layer.id,
|
||||
title: layer.title,
|
||||
fill: qgisRgba(layer.fill),
|
||||
outline: qgisRgba(layer.outline, layer.outlineAlpha ?? 255),
|
||||
outlineWidth: String(layer.outlineWidth),
|
||||
}));
|
||||
}
|
||||
|
||||
function makeQgisScript(options) {
|
||||
return `
|
||||
from pathlib import Path
|
||||
@@ -1384,6 +1345,7 @@ PROJECT_PATH = ${JSON.stringify(options.projectPath)}
|
||||
PREVIEW_PATH = ${JSON.stringify(options.previewPath)}
|
||||
PREVIEW_EXTENT = [${options.previewExtent.split(",").map(Number).join(", ")}]
|
||||
LAYER_PREFIX = ${JSON.stringify(options.layerPrefix || "osm2streets")}
|
||||
LAYER_SPECS = ${JSON.stringify(qgisLayerSpecs(), null, 4)}
|
||||
|
||||
def fill_symbol(color, outline="0,0,0,0", outline_width="0"):
|
||||
return QgsFillSymbol.createSimple({
|
||||
@@ -1412,17 +1374,16 @@ project.setCrs(QgsCoordinateReferenceSystem("EPSG:4326"))
|
||||
project.setPresetHomePath(str(Path(PROJECT_PATH).parent))
|
||||
|
||||
layers = {
|
||||
"road_surface": make_layer("road_surface", f"{LAYER_PREFIX} road surface", "43,43,40,255", "30,30,28,255", "0.04"),
|
||||
"intersection_surface": make_layer("intersection_surface", f"{LAYER_PREFIX} intersection surface", "43,43,40,255", "30,30,28,255", "0.04"),
|
||||
"sidewalks": make_layer("sidewalks", f"{LAYER_PREFIX} sidewalks", "190,190,182,255", "156,156,148,255", "0.025"),
|
||||
"sidewalk_corners": make_layer("sidewalk_corners", f"{LAYER_PREFIX} sidewalk corners", "190,190,182,255", "156,156,148,255", "0.025"),
|
||||
"crosswalks": make_layer("crosswalks", f"{LAYER_PREFIX} crosswalks", "255,255,246,255"),
|
||||
"lane_separators": make_layer("lane_separators", f"{LAYER_PREFIX} lane separators", "238,238,230,255"),
|
||||
"center_lines": make_layer("center_lines", f"{LAYER_PREFIX} center lines", "245,190,42,255"),
|
||||
"vehicle_stop_lines": make_layer("vehicle_stop_lines", f"{LAYER_PREFIX} vehicle stop lines", "255,255,246,255"),
|
||||
"lane_arrows": make_layer("lane_arrows_webscale", f"{LAYER_PREFIX} lane arrows", "255,255,246,255", "43,43,40,200", "0.015"),
|
||||
spec["id"]: make_layer(
|
||||
spec["id"],
|
||||
f"{LAYER_PREFIX} {spec['title']}",
|
||||
spec["fill"],
|
||||
spec["outline"],
|
||||
spec["outlineWidth"],
|
||||
)
|
||||
for spec in LAYER_SPECS
|
||||
}
|
||||
draw_order = ["road_surface", "intersection_surface", "sidewalks", "sidewalk_corners", "lane_separators", "center_lines", "crosswalks", "vehicle_stop_lines", "lane_arrows"]
|
||||
draw_order = [spec["id"] for spec in LAYER_SPECS]
|
||||
for key in draw_order:
|
||||
project.addMapLayer(layers[key], False)
|
||||
root = project.layerTreeRoot()
|
||||
|
||||
164
scripts/lib/scene-layers.js
Normal file
164
scripts/lib/scene-layers.js
Normal file
@@ -0,0 +1,164 @@
|
||||
"use strict";
|
||||
|
||||
// Single source of truth for the osm2streets render layers.
|
||||
//
|
||||
// The same nine layers, in the same order, previously appeared four times:
|
||||
// the merged-scene z_index table, the scene style JSON, the generated QGIS
|
||||
// project (layer dict + draw_order), and the README's manual rebuild snippet.
|
||||
// Adding a layer or changing a z-index meant editing all of them in lockstep,
|
||||
// and a missed copy produces a silently mis-stacked scene downstream in
|
||||
// Blender/Cesium. Everything now derives from SCENE_LAYERS.
|
||||
//
|
||||
// zIndex doubles as draw order: lowest paints first (bottom of the stack).
|
||||
// outline: null means "no stroke" (QGIS gets a fully transparent outline).
|
||||
|
||||
const SCENE_LAYERS = [
|
||||
{
|
||||
id: "road_surface",
|
||||
splitKey: "roadSurface",
|
||||
zIndex: 10,
|
||||
title: "road surface",
|
||||
fill: "#2b2b28",
|
||||
outline: "#1e1e1c",
|
||||
outlineWidth: 0.04,
|
||||
},
|
||||
{
|
||||
id: "intersection_surface",
|
||||
splitKey: "intersectionSurface",
|
||||
zIndex: 20,
|
||||
title: "intersection surface",
|
||||
fill: "#2b2b28",
|
||||
outline: "#1e1e1c",
|
||||
outlineWidth: 0.04,
|
||||
},
|
||||
{
|
||||
id: "sidewalks",
|
||||
splitKey: "sidewalks",
|
||||
zIndex: 30,
|
||||
title: "sidewalks",
|
||||
fill: "#bebeb6",
|
||||
outline: "#9c9c94",
|
||||
outlineWidth: 0.025,
|
||||
},
|
||||
{
|
||||
id: "sidewalk_corners",
|
||||
splitKey: "sidewalkCorners",
|
||||
zIndex: 40,
|
||||
title: "sidewalk corners",
|
||||
fill: "#bebeb6",
|
||||
outline: "#9c9c94",
|
||||
outlineWidth: 0.025,
|
||||
},
|
||||
{
|
||||
id: "lane_separators",
|
||||
splitKey: "laneSeparators",
|
||||
zIndex: 50,
|
||||
title: "lane separators",
|
||||
fill: "#eeeee6",
|
||||
outline: null,
|
||||
outlineWidth: 0,
|
||||
},
|
||||
{
|
||||
id: "center_lines",
|
||||
splitKey: "centerLines",
|
||||
zIndex: 60,
|
||||
title: "center lines",
|
||||
fill: "#f5be2a",
|
||||
outline: null,
|
||||
outlineWidth: 0,
|
||||
},
|
||||
{
|
||||
id: "crosswalks",
|
||||
splitKey: "crosswalks",
|
||||
zIndex: 70,
|
||||
title: "crosswalks",
|
||||
fill: "#fffff6",
|
||||
outline: null,
|
||||
outlineWidth: 0,
|
||||
},
|
||||
{
|
||||
id: "vehicle_stop_lines",
|
||||
splitKey: "vehicleStopLines",
|
||||
zIndex: 80,
|
||||
title: "vehicle stop lines",
|
||||
fill: "#fffff6",
|
||||
outline: null,
|
||||
outlineWidth: 0,
|
||||
},
|
||||
{
|
||||
id: "lane_arrows_webscale",
|
||||
splitKey: "laneArrows",
|
||||
zIndex: 90,
|
||||
title: "lane arrows",
|
||||
fill: "#fffff6",
|
||||
outline: "#2b2b28",
|
||||
outlineAlpha: 200,
|
||||
outlineWidth: 0.015,
|
||||
},
|
||||
];
|
||||
|
||||
const SCENE_FILE = "osm2streets_scene.geojson";
|
||||
const SCENE_STYLE_FILE = "osm2streets_scene_style.json";
|
||||
|
||||
function layerFile(layer) {
|
||||
return `${layer.id}.geojson`;
|
||||
}
|
||||
|
||||
// getCollection(layer) -> FeatureCollection, so callers can source layers from
|
||||
// the in-memory split (build) or from disk (reimport) with the same merge.
|
||||
function mergeScene(getCollection) {
|
||||
return {
|
||||
type: "FeatureCollection",
|
||||
features: SCENE_LAYERS.flatMap((layer) => {
|
||||
const collection = getCollection(layer) || {};
|
||||
return (collection.features || []).map((feature) => ({
|
||||
...feature,
|
||||
properties: {
|
||||
...(feature.properties || {}),
|
||||
render_layer: layer.id,
|
||||
z_index: layer.zIndex,
|
||||
},
|
||||
}));
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function sceneStyle() {
|
||||
return {
|
||||
version: 1,
|
||||
geometry: "polygon",
|
||||
sortProperty: "z_index",
|
||||
layerProperty: "render_layer",
|
||||
layers: SCENE_LAYERS.map((layer) => ({
|
||||
id: layer.id,
|
||||
zIndex: layer.zIndex,
|
||||
fill: layer.fill,
|
||||
outline: layer.outline,
|
||||
outlineWidth: layer.outlineWidth,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
// QGIS symbol properties want "r,g,b,a" strings rather than hex.
|
||||
function qgisRgba(hex, alpha = 255) {
|
||||
if (!hex) return "0,0,0,0";
|
||||
const match = /^#?([0-9a-f]{6})$/i.exec(hex.trim());
|
||||
if (!match) {
|
||||
throw new Error(`Expected #rrggbb color, got: ${hex}`);
|
||||
}
|
||||
const value = parseInt(match[1], 16);
|
||||
const r = (value >> 16) & 0xff;
|
||||
const g = (value >> 8) & 0xff;
|
||||
const b = value & 0xff;
|
||||
return `${r},${g},${b},${alpha}`;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
SCENE_LAYERS,
|
||||
SCENE_FILE,
|
||||
SCENE_STYLE_FILE,
|
||||
layerFile,
|
||||
mergeScene,
|
||||
sceneStyle,
|
||||
qgisRgba,
|
||||
};
|
||||
179
scripts/reimport-gpkg.js
Executable file
179
scripts/reimport-gpkg.js
Executable file
@@ -0,0 +1,179 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
"use strict";
|
||||
|
||||
// Reverse of the intermediates stage: pull manually-edited layers back out of
|
||||
// <area>.gpkg into osm2streets_web_out/*.geojson and rebuild the merged scene.
|
||||
//
|
||||
// Use after hand-fixing geometry in QGIS. Re-running intermediates would
|
||||
// regenerate the GeoPackage from OSM and throw those edits away.
|
||||
//
|
||||
// Every layer is exported to a staging directory and parsed before anything in
|
||||
// outDir is touched: ogr2ogr exits non-zero on a missing layer but still leaves
|
||||
// a zero-byte file behind, so a partial export must not reach the output tree.
|
||||
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const os = require("os");
|
||||
const { execFileSync } = require("child_process");
|
||||
const {
|
||||
SCENE_LAYERS,
|
||||
SCENE_FILE,
|
||||
SCENE_STYLE_FILE,
|
||||
layerFile,
|
||||
mergeScene,
|
||||
sceneStyle,
|
||||
} = require("./lib/scene-layers");
|
||||
|
||||
const args = parseArgs(process.argv.slice(2));
|
||||
const config = loadConfig(args);
|
||||
const qgisApp = config.qgisApp || "/Applications/QGIS.app";
|
||||
const qgisMacOS = path.join(qgisApp, "Contents", "MacOS");
|
||||
const ogr2ogr = path.join(qgisMacOS, "ogr2ogr");
|
||||
const ogrinfo = path.join(qgisMacOS, "ogrinfo");
|
||||
const outDir = path.resolve(requireText(config.outDir, "outDir"));
|
||||
const gpkgPath = path.resolve(requireText(config.gpkg, "gpkg"));
|
||||
|
||||
for (const exe of [ogr2ogr, ogrinfo]) {
|
||||
if (!fs.existsSync(exe)) {
|
||||
throw new Error(`QGIS executable not found: ${exe}`);
|
||||
}
|
||||
}
|
||||
if (!fs.existsSync(gpkgPath)) {
|
||||
throw new Error(`GeoPackage not found: ${gpkgPath}\nRun the intermediates stage first.`);
|
||||
}
|
||||
if (!fs.existsSync(outDir)) {
|
||||
throw new Error(`GeoJSON output directory not found: ${outDir}`);
|
||||
}
|
||||
|
||||
console.log(`Reimport: ${gpkgPath}`);
|
||||
console.log(`Target: ${outDir}`);
|
||||
|
||||
const present = gpkgLayers();
|
||||
const missing = SCENE_LAYERS.filter((layer) => !present.has(layer.id)).map((layer) => layer.id);
|
||||
if (missing.length) {
|
||||
throw new Error(
|
||||
`GeoPackage is missing ${missing.length} layer(s): ${missing.join(", ")}\n` +
|
||||
`Present: ${[...present].join(", ") || "(none)"}`,
|
||||
);
|
||||
}
|
||||
|
||||
const stagingDir = fs.mkdtempSync(path.join(os.tmpdir(), "osm2streets-reimport-"));
|
||||
try {
|
||||
const staged = SCENE_LAYERS.map((layer) => {
|
||||
const stagedPath = path.join(stagingDir, layerFile(layer));
|
||||
exportLayer(layer.id, stagedPath);
|
||||
const collection = readCollection(stagedPath, layer.id);
|
||||
console.log(`${layer.id}\tfeatures=${collection.features.length}`);
|
||||
return { layer, stagedPath, collection };
|
||||
});
|
||||
|
||||
for (const item of staged) {
|
||||
// Copy rather than rename: the staging dir may be on another filesystem.
|
||||
fs.copyFileSync(item.stagedPath, path.join(outDir, layerFile(item.layer)));
|
||||
}
|
||||
|
||||
const byId = new Map(staged.map((item) => [item.layer.id, item.collection]));
|
||||
const scene = mergeScene((layer) => byId.get(layer.id));
|
||||
fs.writeFileSync(path.join(outDir, SCENE_FILE), JSON.stringify(scene));
|
||||
fs.writeFileSync(
|
||||
path.join(outDir, SCENE_STYLE_FILE),
|
||||
JSON.stringify(sceneStyle(), null, 2),
|
||||
);
|
||||
console.log(`${path.join(outDir, SCENE_FILE)}\tfeatures=${scene.features.length}`);
|
||||
|
||||
const empty = staged.filter((item) => item.collection.features.length === 0);
|
||||
if (empty.length) {
|
||||
console.warn(`Warning: empty layer(s): ${empty.map((item) => item.layer.id).join(", ")}`);
|
||||
}
|
||||
} finally {
|
||||
fs.rmSync(stagingDir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
function parseArgs(argv) {
|
||||
const out = {};
|
||||
for (let i = 0; i < argv.length; i += 1) {
|
||||
const arg = argv[i];
|
||||
if (!arg.startsWith("--")) continue;
|
||||
const key = arg.slice(2).replace(/-([a-z])/g, (_, c) => c.toUpperCase());
|
||||
const next = argv[i + 1];
|
||||
if (!next || next.startsWith("--")) {
|
||||
out[key] = "true";
|
||||
} else {
|
||||
out[key] = next;
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function loadConfig(cliArgs) {
|
||||
const base = {};
|
||||
if (cliArgs.config) {
|
||||
const file = path.resolve(cliArgs.config);
|
||||
if (!fs.existsSync(file)) {
|
||||
throw new Error(`Config file not found: ${file}`);
|
||||
}
|
||||
Object.assign(base, JSON.parse(fs.readFileSync(file, "utf8")));
|
||||
}
|
||||
for (const key of ["qgisApp", "outDir", "gpkg"]) {
|
||||
if (cliArgs[key] !== undefined) base[key] = cliArgs[key];
|
||||
}
|
||||
return base;
|
||||
}
|
||||
|
||||
function requireText(value, key) {
|
||||
if (typeof value !== "string" || value.trim() === "") {
|
||||
throw new Error(`Missing config key: ${key}`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function gdalEnv() {
|
||||
return {
|
||||
PROJ_LIB: path.join(qgisApp, "Contents", "Resources", "qgis", "proj"),
|
||||
GDAL_DATA: path.join(qgisApp, "Contents", "Resources", "qgis", "gdal"),
|
||||
};
|
||||
}
|
||||
|
||||
function gpkgLayers() {
|
||||
const output = execFileSync(ogrinfo, ["-q", gpkgPath], {
|
||||
encoding: "utf8",
|
||||
env: { ...process.env, ...gdalEnv() },
|
||||
});
|
||||
const names = new Set();
|
||||
for (const line of output.split("\n")) {
|
||||
const match = /^\s*\d+:\s+(\S+)/.exec(line);
|
||||
if (match) names.add(match[1]);
|
||||
}
|
||||
return names;
|
||||
}
|
||||
|
||||
function exportLayer(layerName, destination) {
|
||||
// No COORDINATE_PRECISION here on purpose: the default already round-trips
|
||||
// full double precision, and setting it explicitly makes GDAL run its
|
||||
// precision-reduction pass, which drops vertices that collapse at the given
|
||||
// resolution (measured: 28 points lost across 7 lane-arrow polygons).
|
||||
execFileSync(ogr2ogr, [
|
||||
"-f", "GeoJSON",
|
||||
destination,
|
||||
gpkgPath,
|
||||
layerName,
|
||||
], {
|
||||
stdio: "inherit",
|
||||
env: { ...process.env, ...gdalEnv() },
|
||||
});
|
||||
}
|
||||
|
||||
function readCollection(file, layerName) {
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(fs.readFileSync(file, "utf8"));
|
||||
} catch (error) {
|
||||
throw new Error(`Layer '${layerName}' did not export valid GeoJSON: ${error.message}`);
|
||||
}
|
||||
if (parsed.type !== "FeatureCollection" || !Array.isArray(parsed.features)) {
|
||||
throw new Error(`Layer '${layerName}' did not export a FeatureCollection`);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
Reference in New Issue
Block a user