feat(pipeline): make compressed assets the default delivery
This commit is contained in:
@@ -54,7 +54,7 @@ cp config/examples/template.json config/areas/my-area.json
|
||||
| `turnLaneArrows` | | 见下 | 从 OSM `turn:lanes:*` 生成自定义车道箭头的发布开关 |
|
||||
| `osm2streets` | | 见下 | 透传给 osm2streets 的选项 |
|
||||
| `blender` | | 见下 | Blender 侧选项 |
|
||||
| `compress` | | 见下 | 显式 `compress` 阶段的 GLB 压缩选项 |
|
||||
| `compress` | | 见下 | 默认交付压缩阶段的 GLB 压缩选项 |
|
||||
| `budget` | | 见下 | 区域 GLB 性能与体量预算 |
|
||||
| `outputs` | | 从 `id` 推导 | 输出路径覆盖,逃生舱 |
|
||||
|
||||
@@ -69,11 +69,10 @@ cp config/examples/template.json config/areas/my-area.json
|
||||
| `blender` | `true` | |
|
||||
| `cesium` | `true` | |
|
||||
|
||||
`reimport`、`preview` 和 `compress` **在这里配也没用**——`normalizeAreaConfig` 把它们
|
||||
硬编码为 `false`,只能靠 `--stages` 显式请求。
|
||||
`reimport` 和 `preview` **在这里配也没用**——`normalizeAreaConfig` 把它们
|
||||
硬编码为 `false`,只能靠 `--stages` 显式请求。`compress` 始终默认开启。
|
||||
|
||||
> 恢复动作(reimport)、补丁动作(preview)和替代产物动作(compress)不该被一份
|
||||
> 配置文件变成默认行为。
|
||||
> 恢复动作(reimport)和补丁动作(preview)不该被一份配置文件变成默认行为;压缩是标准交付链的一部分。
|
||||
|
||||
`--stages` 会整体覆盖这里的默认值。
|
||||
|
||||
@@ -133,8 +132,8 @@ cp config/examples/template.json config/areas/my-area.json
|
||||
|
||||
### `compress`
|
||||
|
||||
只影响显式 `--stages compress`。默认压缩链是 texture resize + WebP transcode,
|
||||
不覆盖默认 `<area-id>.glb`。
|
||||
完整构建和显式 `--stages compress` 都使用此配置。默认压缩链是 texture resize + WebP
|
||||
transcode,成功后覆盖标准 `<area-id>.glb`、`.json` 和预览 HTML;未压缩源只保留在构建临时目录。
|
||||
|
||||
| 字段 | 默认 | 说明 |
|
||||
|---|---|---|
|
||||
@@ -177,7 +176,6 @@ cp config/examples/template.json config/areas/my-area.json
|
||||
|
||||
可覆盖的键(`scripts/lib/area-config.js`):`areaDir`、`fileStem`、`geojsonDir`、`gpkg`、
|
||||
`qgisProject`、`qgisPreview`、`blend`、`render`、`glb`、`metadata`、`cesiumPreview`、
|
||||
`compressedFileStem`、`compressedGlb`、`compressedMetadata`、`compressedCesiumPreview`、
|
||||
`vehicleRoute`、`vehicleModel`、`pipelineDir`、`stageManifestDir`。
|
||||
|
||||
**优先改 `fileStem` 或 `areaDir`**——它们能一次性影响全部派生路径。逐个覆盖容易漏。
|
||||
@@ -221,7 +219,7 @@ cp config/examples/template.json config/areas/my-area.json
|
||||
| 布尔字段用 `\|\|` 兜底 | `false` 被翻转 |
|
||||
| 给新字段造顶层平铺别名 | 扩大历史包袱 |
|
||||
| 逐个覆盖 `outputs` 而不用 `fileStem` | 漏掉某个产物路径 |
|
||||
| 在 `stages` 里配 `reimport` / `preview` / `compress` | 无效,被硬编码为 false |
|
||||
| 在 `stages` 里配 `reimport` / `preview` | 无效,被硬编码为 false |
|
||||
| 加数值字段不做范围校验 | 错配置在中途才崩,输出已被破坏 |
|
||||
|
||||
---
|
||||
|
||||
@@ -21,13 +21,13 @@
|
||||
全部是 CommonJS(`package.json` 的 `"type": "commonjs"`),无构建步骤、无 TypeScript、
|
||||
零运行时依赖(唯一依赖 `osm2streets-js-node` 只被 `build-osm2streets-qgis.js` 用)。
|
||||
|
||||
## 可选 GLB 压缩阶段
|
||||
## 默认 GLB 压缩阶段
|
||||
|
||||
### 1. Scope / Trigger
|
||||
|
||||
`compress` 是显式请求的替代产物阶段,不属于 `all`,也不覆盖默认 `<area-id>.glb`。
|
||||
它用于在已有 Cesium GLB 上生成并列压缩 GLB、metadata 和预览页。当前主路径是
|
||||
texture resize + WebP transcode。
|
||||
`compress` 是完整构建和 `all` 的默认末端阶段。它在已有 Cesium GLB 上执行 texture
|
||||
resize + WebP transcode,并以标准 `<area-id>.glb`、`.json` 和预览 HTML 替换未压缩导出。
|
||||
未压缩版本只在 `<areaDir>/_pipeline/compress-*` 临时目录中存在,成功或失败后都会清理。
|
||||
|
||||
`scripts/compress-glb.js` 是该阶段调用的低层脚本,也可单独运行做实验。
|
||||
|
||||
@@ -62,15 +62,14 @@ npm run compress:glb -- --input in.glb --output out.glb [options]
|
||||
### 3. Contracts
|
||||
|
||||
- 输入必须是现有 `.glb` 文件;`--output` 必须不同于 `--input`。
|
||||
- 输出是并列压缩 GLB;默认构建产物不被替换。
|
||||
- `compress` 阶段依赖默认 `glb`、`metadata` 和 `cesiumPreview` 已存在。
|
||||
- 低层脚本仍输出到与输入不同的路径;`build-area.js` 使用临时输入/输出路径,避免原地压缩。
|
||||
- `compress` 阶段依赖标准 `glb`、`metadata` 和 `cesiumPreview` 已存在,并在全部临时交付文件
|
||||
生成和校验后以 `renameSync` 替换它们。
|
||||
- 默认链固定为 `gltf-transform resize -> gltf-transform webp`。
|
||||
- `compress` 阶段默认 `textureSize=768`;低层脚本单独运行时默认 `--texture-size 1024`。
|
||||
- `build-area.js` 默认写:
|
||||
- `<fileStem>-compressed-webp768.glb`
|
||||
- `<fileStem>-compressed-webp768.json`
|
||||
- `<fileStem>-compressed-webp768-cesium-preview.html`
|
||||
- 伴生 metadata 的 `asset` 与 `id="main"` 资产 URL 改为压缩 GLB 文件名;其余资产(包括
|
||||
- `build-area.js` 最终只保留 `<fileStem>.glb`、`<fileStem>.json` 和
|
||||
`<fileStem>-cesium-preview.html`;不会写并列 `-compressed-*` 交付物。
|
||||
- 伴生 metadata 的 `asset` 与 `id="main"` 资产 URL 保持为标准 GLB 文件名;其余资产(包括
|
||||
`category="semantic"` 的 Cesium 分类检查 GLB)必须原样保留。
|
||||
- preview HTML 只替换 `window.OSM_ASSET_PREVIEW_CONFIG` 的 `glbName` /
|
||||
`metadataName` 和 loading 文案,不改 preview runtime。
|
||||
@@ -88,13 +87,12 @@ npm run compress:glb -- --input in.glb --output out.glb [options]
|
||||
| `gltf-transform` 退出非零 | 抛错并带上 status / signal |
|
||||
| `--preview` 没有 `--metadata` | 抛错,因为 preview 必须指向存在的 metadata |
|
||||
| preview HTML 找不到配置块 | 抛错,不做猜测替换 |
|
||||
| `--stages compress` 但默认 preview 不存在 | `Cesium preview not found: <path>` |
|
||||
| `--stages compress` 但标准 preview 不存在 | `Cesium preview not found: <path>` |
|
||||
|
||||
### 5. Good/Base/Bad Cases
|
||||
|
||||
- Good: `--stages cesium,compress` 先重导默认 GLB,再生成并列压缩产物。
|
||||
- Good: `--texture-size 768 --metadata --preview` 生成压缩 GLB、metadata、HTML,
|
||||
源 GLB 保持不变。
|
||||
- Good: `all` 或默认构建先导出 Cesium,再压缩成标准交付物。
|
||||
- Good: `--stages compress` 对已有标准交付物重新压缩,临时源 GLB 不会留在输出目录。
|
||||
- Base: 只传 `--input --output` 生成压缩 GLB,不生成伴生文件。
|
||||
- Bad: 使用 `--meshopt` 后没有做 Cesium 兼容性验证就当默认产物发布。
|
||||
|
||||
@@ -104,9 +102,9 @@ npm run compress:glb -- --input in.glb --output out.glb [options]
|
||||
- `npm run test:compress-glb`:断言压缩 metadata 只替换主资产,不丢失语义资产。
|
||||
- `node --check scripts/build-area.js`
|
||||
- 对目标区域跑一次 `npm run compress:glb -- ... --metadata --preview`
|
||||
- 对目标区域跑一次 `npm run build:area -- --stages compress`
|
||||
- `node scripts/glb-digest.js <compressed.glb>` 确认可解析结构和扩展
|
||||
- 浏览器/Cesium 预览压缩 HTML,确认 `EXT_texture_webp` 在目标环境可加载
|
||||
- 对目标区域跑一次默认 `npm run build:area`
|
||||
- `node scripts/glb-digest.js <area>.glb` 确认可解析结构和扩展
|
||||
- 浏览器/Cesium 预览标准 HTML,确认 `EXT_texture_webp` 在目标环境可加载
|
||||
|
||||
### 7. Wrong vs Correct
|
||||
|
||||
@@ -116,10 +114,10 @@ Wrong:
|
||||
npm run compress:glb -- --input outputs/a/a.glb --output outputs/a/a.glb
|
||||
```
|
||||
|
||||
Correct:
|
||||
Correct for the low-level script:
|
||||
|
||||
```bash
|
||||
npm run compress:glb -- --input outputs/a/a.glb --output outputs/a/a-compressed-webp768.glb
|
||||
npm run compress:glb -- --input outputs/a/a.glb --output /tmp/a-compressed.glb
|
||||
```
|
||||
|
||||
---
|
||||
@@ -758,14 +756,11 @@ runtime 与 HTML,并写 preview manifest。预览内容实现不得回流到
|
||||
|
||||
`compress` manifest:
|
||||
|
||||
- `inputs.glb`
|
||||
- `inputs.metadata`
|
||||
- `inputs.cesiumPreview`
|
||||
- `outputs.compressedGlb`
|
||||
- `outputs.compressedMetadata`
|
||||
- `outputs.compressedCesiumPreview`
|
||||
- `outputs.glb`
|
||||
- `outputs.metadata`
|
||||
- `outputs.cesiumPreview`
|
||||
- `summary.sourceGlb`
|
||||
- `summary.compressedGlb`
|
||||
- `summary.glb`
|
||||
- `summary.options`
|
||||
- `summary.compressionRatio`
|
||||
- `summary.savedBytes`
|
||||
@@ -931,7 +926,7 @@ gpkg: path.resolve(outputOverrides.gpkg || path.join(areaDir, `${fileStem}.gpkg`
|
||||
| `blender` | OSM + GeoJSON → 场景 | `.osm`、`osm2streets_web_out/` | `.blend`、`.png` |
|
||||
| `cesium` | 场景 → GLB + 元数据 + 预览页 | `.blend` | `.glb`、`.json`、预览 HTML 及其静态资源 |
|
||||
| `preview` | 只补生成预览页 | `.glb`、`.json` | 预览 HTML 及其静态资源 |
|
||||
| `compress` | 生成并列压缩 Cesium 产物 | `.glb`、`.json`、默认预览 HTML | `-compressed-webp*.glb/json/html` |
|
||||
| `compress` | 将 Cesium 临时导出替换为压缩交付产物 | 标准 `.glb`、`.json`、预览 HTML | 标准 `.glb/json/html` |
|
||||
|
||||
调度是顶层的阶段 `if`(`build-area.js` 开头),顺序固定,**阶段之间不传内存状态,
|
||||
只通过磁盘产物耦合**。这就是单跑某个阶段能work 的原因。
|
||||
@@ -952,11 +947,11 @@ gpkg: path.resolve(outputOverrides.gpkg || path.join(areaDir, `${fileStem}.gpkg`
|
||||
```js
|
||||
// 'reimport' is deliberately absent from 'all': it is a recovery step for
|
||||
// hand-edited GeoPackages, never part of a full build. build-area.js:159-160
|
||||
all: ["intermediates", "blender", "cesium"],
|
||||
all: ["intermediates", "blender", "cesium", "compress"],
|
||||
```
|
||||
|
||||
`preview` 同样不在 `all` 里——`cesium` 已经包含它。`compress` 也不在 `all` 里——
|
||||
它生成的是替代压缩产物,不是 baseline GLB。
|
||||
`preview` 不在 `all` 里——`cesium` 已经包含它。`compress` 在 `all` 里,确保完整构建
|
||||
以压缩后的标准路径交付。
|
||||
|
||||
### `intermediates` 与 `reimport` 互斥
|
||||
|
||||
@@ -966,9 +961,9 @@ all: ["intermediates", "blender", "cesium"],
|
||||
|
||||
这是**显式拒绝而不是警告**——两者同时开,无论谁先跑,另一个的工作都白做。
|
||||
|
||||
`normalizeAreaConfig` 里 `stages.reimport`、`stages.preview` 和 `stages.compress` 硬编码为 `false`,
|
||||
**不能从配置文件打开**,只能靠 `--stages` 显式请求。
|
||||
恢复动作、补丁动作和替代产物动作都不该被一份配置文件变成默认行为。
|
||||
`normalizeAreaConfig` 里 `stages.reimport` 和 `stages.preview` 硬编码为 `false`,
|
||||
只能靠 `--stages` 显式请求;`stages.compress` 为默认 `true`。
|
||||
恢复动作和补丁动作不该被一份配置文件变成默认行为,压缩则属于标准交付链。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -60,7 +60,7 @@ config/areas/<id>.json
|
||||
│ + 车辆巡航路线与模型
|
||||
│ → _pipeline/stages/preview.manifest.json
|
||||
│
|
||||
└─[compress]──────▶ 生成并列压缩 GLB / metadata / preview
|
||||
└─[compress]──────▶ 临时保存未压缩导出,再以标准 GLB / metadata / preview 路径交付压缩版本
|
||||
→ _pipeline/stages/compress.manifest.json
|
||||
```
|
||||
|
||||
|
||||
21
README.md
21
README.md
@@ -8,11 +8,10 @@
|
||||
|
||||
- `<area-id>.blend`:Blender 场景,包含道路、建筑、水体、植被等
|
||||
- `<area-id>.png`:Blender 预览渲染
|
||||
- `<area-id>.glb`:Cesium 可加载的完整 3D 模型(基线产物)
|
||||
- `<area-id>.glb`:Cesium 可加载的完整 3D 模型(默认压缩交付产物)
|
||||
- `<area-id>-roads.glb`、`-buildings.glb`、`-vegetation.glb`、`-water.glb`:Cesium 分类检查用的辅助模型
|
||||
- `<area-id>.json`:Cesium 放置元数据和示例代码
|
||||
- `<area-id>-cesium-preview.html`:Cesium 本地预览页
|
||||
- `<area-id>-compressed-webp768.glb/json/html`:显式 `compress` 阶段生成的可选压缩预览产物
|
||||
- `osm2streets_web_out/`:osm2streets GeoJSON 中间层
|
||||
- `<area-id>.gpkg` / `<area-id>.qgz` / `<area-id>-preview.png`:QGIS 调试资产
|
||||
|
||||
@@ -62,15 +61,14 @@ npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages reimport
|
||||
```
|
||||
|
||||
`intermediates` 会生成 osm2streets GeoJSON、GeoPackage、QGIS 工程和 QGIS 预览图。`blender` 使用 OSM 和 osm2streets GeoJSON 生成 `.blend`/`.png`。`cesium` 从 `.blend` 导出 `.glb`/`.json`,并生成 Cesium 预览 HTML。`preview` 只在已有 `.glb/.json` 时补生成 HTML。`compress` 从已有 `.glb/.json/html` 生成并列压缩产物,不覆盖默认 GLB。`reimport` 把手工编辑过的 GeoPackage 回导为 GeoJSON,不含在 `all` 里,详见 [QGIS 手工修正工作流](#qgis-手工修正工作流)。
|
||||
`intermediates` 会生成 osm2streets GeoJSON、GeoPackage、QGIS 工程和 QGIS 预览图。`blender` 使用 OSM 和 osm2streets GeoJSON 生成 `.blend`/`.png`。`cesium` 从 `.blend` 导出 `.glb`/`.json`,并生成 Cesium 预览 HTML。完整构建会接着运行 `compress`,以标准 `.glb/.json/html` 名称交付压缩版本;未压缩导出仅存在于构建时临时目录。`preview` 只在已有 `.glb/.json` 时补生成 HTML。`reimport` 把手工编辑过的 GeoPackage 回导为 GeoJSON,不含在 `all` 里,详见 [QGIS 手工修正工作流](#qgis-手工修正工作流)。
|
||||
|
||||
Cesium 预览默认显示完整场景。点击 `Inspect` 后会按道路、建筑、绿化与设施、水体加载辅助 GLB;该模式用于单独检查生成结果,主 GLB 仍是完整场景和下游使用的基线。
|
||||
压缩预览同样保留这些分类检查资产;重跑 `compress` 后应打开新生成的压缩预览 HTML。
|
||||
Cesium 预览默认显示完整场景。点击 `Inspect` 后会按道路、建筑、绿化与设施、水体加载辅助 GLB;该模式用于单独检查生成结果,主 GLB 是压缩后的下游交付资产。
|
||||
|
||||
`compress` 不含在 `all` 里,也不能从配置文件默认开启。需要重导出 Cesium 后立刻生成压缩产物时,显式跑:
|
||||
`compress` 包含在完整构建的默认阶段和 `all` 中。需要对已有标准产物重新压缩时,可以显式跑:
|
||||
|
||||
```bash
|
||||
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages cesium,compress
|
||||
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages compress
|
||||
```
|
||||
|
||||
## 区域诊断
|
||||
@@ -211,8 +209,7 @@ QGIS road-layer knobs:
|
||||
"areaDir": "/absolute/path/to/custom-area",
|
||||
"blend": "/absolute/path/to/custom.blend",
|
||||
"glb": "/absolute/path/to/custom.glb",
|
||||
"cesiumPreview": "/absolute/path/to/custom-preview.html",
|
||||
"compressedGlb": "/absolute/path/to/custom-compressed.glb"
|
||||
"cesiumPreview": "/absolute/path/to/custom-preview.html"
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -277,8 +274,8 @@ Cesium GLB 低层导出:
|
||||
|
||||
## 压缩 GLB
|
||||
|
||||
推荐使用显式 `compress` 阶段。它不会覆盖默认 `<area-id>.glb`,只生成并列的压缩 GLB、
|
||||
metadata 和预览页:
|
||||
完整构建默认运行 `compress`。它将未压缩 Cesium 导出复制到 `_pipeline` 下的临时目录,
|
||||
压缩成功后才替换标准 `<area-id>.glb`、`.json` 和预览页;最终输出目录不保留并列压缩文件:
|
||||
|
||||
```bash
|
||||
npm run build:area -- \
|
||||
@@ -291,7 +288,7 @@ npm run build:area -- \
|
||||
```bash
|
||||
npm run compress:glb -- \
|
||||
--input outputs/nantaizi-lake-innovation-valley/nantaizi-lake-innovation-valley.glb \
|
||||
--output outputs/nantaizi-lake-innovation-valley/nantaizi-lake-innovation-valley-compressed-webp768.glb \
|
||||
--output /tmp/nantaizi-lake-innovation-valley-compressed.glb \
|
||||
--texture-size 768 \
|
||||
--metadata outputs/nantaizi-lake-innovation-valley/nantaizi-lake-innovation-valley.json \
|
||||
--preview outputs/nantaizi-lake-innovation-valley/nantaizi-lake-innovation-valley-cesium-preview.html
|
||||
|
||||
@@ -347,74 +347,82 @@ function compressCesiumGlb(area) {
|
||||
ensureFile(area.outputs.metadata, "Cesium metadata");
|
||||
ensureFile(area.outputs.cesiumPreview, "Cesium preview");
|
||||
ensureFile(path.join(repoRoot, "scripts", "compress-glb.js"), "GLB compressor");
|
||||
fs.mkdirSync(path.dirname(area.outputs.compressedGlb), { recursive: true });
|
||||
fs.mkdirSync(path.dirname(area.outputs.compressedMetadata), { recursive: true });
|
||||
fs.mkdirSync(path.dirname(area.outputs.compressedCesiumPreview), { recursive: true });
|
||||
|
||||
const compressArgs = [
|
||||
path.join(repoRoot, "scripts", "compress-glb.js"),
|
||||
"--input",
|
||||
area.outputs.glb,
|
||||
"--output",
|
||||
area.outputs.compressedGlb,
|
||||
"--texture-size",
|
||||
String(area.compress.textureSize),
|
||||
"--quality",
|
||||
String(area.compress.quality),
|
||||
"--effort",
|
||||
String(area.compress.effort),
|
||||
"--metadata",
|
||||
area.outputs.metadata,
|
||||
"--metadata-output",
|
||||
area.outputs.compressedMetadata,
|
||||
"--preview",
|
||||
area.outputs.cesiumPreview,
|
||||
"--preview-output",
|
||||
area.outputs.compressedCesiumPreview,
|
||||
];
|
||||
if (area.compress.meshopt) {
|
||||
compressArgs.push("--meshopt");
|
||||
}
|
||||
|
||||
console.log("Stage: compress (Cesium GLB texture resize + WebP)");
|
||||
const started = Date.now();
|
||||
const startedAt = new Date(started).toISOString();
|
||||
runCommand(process.execPath, compressArgs, "compress");
|
||||
const finished = Date.now();
|
||||
const sourceDigest = glbDigest(area.outputs.glb);
|
||||
const compressedDigest = glbDigest(area.outputs.compressedGlb);
|
||||
writeStageManifest(area, {
|
||||
stage: "compress",
|
||||
status: "ok",
|
||||
config: configPath,
|
||||
startedAt,
|
||||
finishedAt: new Date(finished).toISOString(),
|
||||
durationMs: finished - started,
|
||||
inputs: {
|
||||
glb: fileRecord(area.outputs.glb),
|
||||
metadata: fileRecord(area.outputs.metadata),
|
||||
cesiumPreview: fileRecord(area.outputs.cesiumPreview),
|
||||
},
|
||||
outputs: {
|
||||
compressedGlb: fileRecord(area.outputs.compressedGlb),
|
||||
compressedMetadata: fileRecord(area.outputs.compressedMetadata),
|
||||
compressedCesiumPreview: fileRecord(area.outputs.compressedCesiumPreview),
|
||||
},
|
||||
summary: {
|
||||
sourceGlb: glbSummary(sourceDigest),
|
||||
compressedGlb: glbSummary(compressedDigest),
|
||||
budget: evaluateGlbBudget(compressedDigest, area.budget),
|
||||
options: {
|
||||
textureSize: area.compress.textureSize,
|
||||
quality: area.compress.quality,
|
||||
effort: area.compress.effort,
|
||||
meshopt: area.compress.meshopt,
|
||||
fs.mkdirSync(area.outputs.pipelineDir, { recursive: true });
|
||||
const temporaryDir = fs.mkdtempSync(path.join(area.outputs.pipelineDir, "compress-"));
|
||||
|
||||
// The compressor rejects an in-place transform. Preserve the exporter output
|
||||
// in a temporary directory, then atomically promote the compressed delivery.
|
||||
const sourceDir = path.join(temporaryDir, "source");
|
||||
const outputDir = path.join(temporaryDir, "delivery");
|
||||
const stagedSourceGlb = path.join(sourceDir, path.basename(area.outputs.glb));
|
||||
const stagedSourceMetadata = path.join(sourceDir, path.basename(area.outputs.metadata));
|
||||
const stagedSourcePreview = path.join(sourceDir, path.basename(area.outputs.cesiumPreview));
|
||||
const stagedDeliveryGlb = path.join(outputDir, path.basename(area.outputs.glb));
|
||||
const stagedDeliveryMetadata = path.join(outputDir, path.basename(area.outputs.metadata));
|
||||
const stagedDeliveryPreview = path.join(outputDir, path.basename(area.outputs.cesiumPreview));
|
||||
try {
|
||||
fs.mkdirSync(sourceDir, { recursive: true });
|
||||
fs.copyFileSync(area.outputs.glb, stagedSourceGlb);
|
||||
fs.copyFileSync(area.outputs.metadata, stagedSourceMetadata);
|
||||
fs.copyFileSync(area.outputs.cesiumPreview, stagedSourcePreview);
|
||||
const sourceDigest = glbDigest(stagedSourceGlb);
|
||||
const compressArgs = [
|
||||
path.join(repoRoot, "scripts", "compress-glb.js"),
|
||||
"--input", stagedSourceGlb,
|
||||
"--output", stagedDeliveryGlb,
|
||||
"--texture-size", String(area.compress.textureSize),
|
||||
"--quality", String(area.compress.quality),
|
||||
"--effort", String(area.compress.effort),
|
||||
"--metadata", stagedSourceMetadata,
|
||||
"--metadata-output", stagedDeliveryMetadata,
|
||||
"--preview", stagedSourcePreview,
|
||||
"--preview-output", stagedDeliveryPreview,
|
||||
];
|
||||
if (area.compress.meshopt) compressArgs.push("--meshopt");
|
||||
|
||||
console.log("Stage: compress (Cesium GLB texture resize + WebP)");
|
||||
runCommand(process.execPath, compressArgs, "compress");
|
||||
ensureFile(stagedDeliveryGlb, "Compressed delivery GLB");
|
||||
ensureFile(stagedDeliveryMetadata, "Compressed delivery metadata");
|
||||
ensureFile(stagedDeliveryPreview, "Compressed delivery preview");
|
||||
const compressedDigest = glbDigest(stagedDeliveryGlb);
|
||||
fs.renameSync(stagedDeliveryGlb, area.outputs.glb);
|
||||
fs.renameSync(stagedDeliveryMetadata, area.outputs.metadata);
|
||||
fs.renameSync(stagedDeliveryPreview, area.outputs.cesiumPreview);
|
||||
const finished = Date.now();
|
||||
writeStageManifest(area, {
|
||||
stage: "compress",
|
||||
status: "ok",
|
||||
config: configPath,
|
||||
startedAt,
|
||||
finishedAt: new Date(finished).toISOString(),
|
||||
durationMs: finished - started,
|
||||
inputs: {},
|
||||
outputs: {
|
||||
glb: fileRecord(area.outputs.glb),
|
||||
metadata: fileRecord(area.outputs.metadata),
|
||||
cesiumPreview: fileRecord(area.outputs.cesiumPreview),
|
||||
},
|
||||
compressionRatio: Number((compressedDigest.fileBytes / sourceDigest.fileBytes).toFixed(4)),
|
||||
savedBytes: sourceDigest.fileBytes - compressedDigest.fileBytes,
|
||||
},
|
||||
warnings: glbBudgetWarnings("Compressed", compressedDigest, area.budget),
|
||||
});
|
||||
summary: {
|
||||
sourceGlb: glbSummary(sourceDigest),
|
||||
glb: glbSummary(compressedDigest),
|
||||
budget: evaluateGlbBudget(compressedDigest, area.budget),
|
||||
options: {
|
||||
textureSize: area.compress.textureSize,
|
||||
quality: area.compress.quality,
|
||||
effort: area.compress.effort,
|
||||
meshopt: area.compress.meshopt,
|
||||
},
|
||||
compressionRatio: Number((compressedDigest.fileBytes / sourceDigest.fileBytes).toFixed(4)),
|
||||
savedBytes: sourceDigest.fileBytes - compressedDigest.fileBytes,
|
||||
},
|
||||
warnings: glbBudgetWarnings("Compressed", compressedDigest, area.budget),
|
||||
});
|
||||
} finally {
|
||||
fs.rmSync(temporaryDir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function blenderExecutable(area) {
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
#!/usr/bin/env node
|
||||
"use strict";
|
||||
|
||||
// Experimental GLB compression wrapper. Keeps the source GLB intact and writes
|
||||
// a separate compressed artifact for visual comparison in Cesium.
|
||||
// GLB compression wrapper. The caller supplies distinct input and output files;
|
||||
// build-area.js uses staging files before promoting the compressed delivery.
|
||||
|
||||
const fs = require("fs");
|
||||
const os = require("os");
|
||||
|
||||
@@ -25,8 +25,6 @@ function normalizeAreaConfig(raw, options = {}) {
|
||||
const fileStem = outputOverrides.fileStem || id;
|
||||
const compress = normalizeCompressConfig(raw.compress);
|
||||
const budget = normalizeBudgetConfig(raw.budget);
|
||||
const compressedFileStem = outputOverrides.compressedFileStem ||
|
||||
`${fileStem}-compressed-webp${compress.textureSize}${compress.meshopt ? "-meshopt" : ""}`;
|
||||
const pipelineDir = path.resolve(outputOverrides.pipelineDir || path.join(areaDir, "_pipeline"));
|
||||
const geojsonDir = path.resolve(outputOverrides.geojsonDir || path.join(areaDir, "osm2streets_web_out"));
|
||||
const outputs = {
|
||||
@@ -47,15 +45,6 @@ function normalizeAreaConfig(raw, options = {}) {
|
||||
cesiumPreview: path.resolve(
|
||||
outputOverrides.cesiumPreview || path.join(areaDir, `${fileStem}-cesium-preview.html`),
|
||||
),
|
||||
compressedGlb: path.resolve(
|
||||
outputOverrides.compressedGlb || path.join(areaDir, `${compressedFileStem}.glb`),
|
||||
),
|
||||
compressedMetadata: path.resolve(
|
||||
outputOverrides.compressedMetadata || path.join(areaDir, `${compressedFileStem}.json`),
|
||||
),
|
||||
compressedCesiumPreview: path.resolve(
|
||||
outputOverrides.compressedCesiumPreview || path.join(areaDir, `${compressedFileStem}-cesium-preview.html`),
|
||||
),
|
||||
vehicleRoute: path.resolve(outputOverrides.vehicleRoute || path.join(areaDir, `${fileStem}-vehicle-route.json`)),
|
||||
vehicleModel: path.resolve(outputOverrides.vehicleModel || path.join(areaDir, `${fileStem}-vehicle-car.gltf`)),
|
||||
trafficSignalAssemblies: path.resolve(
|
||||
@@ -80,7 +69,7 @@ function normalizeAreaConfig(raw, options = {}) {
|
||||
cesium: raw.stages?.cesium ?? true,
|
||||
reimport: false,
|
||||
preview: false,
|
||||
compress: false,
|
||||
compress: true,
|
||||
},
|
||||
qgis: {
|
||||
arrowScale: raw.qgis?.arrowScale ?? raw.arrowScale ?? 0.8,
|
||||
|
||||
@@ -334,9 +334,6 @@ function artifactStatus(area) {
|
||||
["Cesium GLB", area.outputs.glb, true, "file"],
|
||||
["Cesium metadata", area.outputs.metadata, true, "file"],
|
||||
["Cesium preview", area.outputs.cesiumPreview, true, "file"],
|
||||
["Compressed GLB", area.outputs.compressedGlb, false, "file"],
|
||||
["Compressed metadata", area.outputs.compressedMetadata, false, "file"],
|
||||
["Compressed preview", area.outputs.compressedCesiumPreview, false, "file"],
|
||||
];
|
||||
return entries.map(([label, file, expected, type]) => {
|
||||
const exists = fs.existsSync(file);
|
||||
@@ -374,6 +371,7 @@ function metadataSummary(file, warnings) {
|
||||
}
|
||||
|
||||
function stageManifestStatus(area, configPath = null) {
|
||||
const compressionComplete = fs.existsSync(stageManifestPath(area, "compress"));
|
||||
const reimportManifest = stageManifestPath(area, "reimport");
|
||||
const hasReimportManifest = fs.existsSync(reimportManifest);
|
||||
const derivedConfig = path.join(area.outputs.pipelineDir, "osm2streets-qgis.config.json");
|
||||
@@ -446,7 +444,7 @@ function stageManifestStatus(area, configPath = null) {
|
||||
inputs: {
|
||||
blend: area.outputs.blend,
|
||||
},
|
||||
outputs: {
|
||||
outputs: compressionComplete ? {} : {
|
||||
glb: area.outputs.glb,
|
||||
metadata: area.outputs.metadata,
|
||||
},
|
||||
@@ -457,15 +455,20 @@ function stageManifestStatus(area, configPath = null) {
|
||||
inputs: {
|
||||
...(configPath ? { config: configPath } : {}),
|
||||
osm: area.input,
|
||||
glb: area.outputs.glb,
|
||||
metadata: area.outputs.metadata,
|
||||
...(compressionComplete ? {} : {
|
||||
glb: area.outputs.glb,
|
||||
metadata: area.outputs.metadata,
|
||||
}),
|
||||
lanePolygons: path.join(area.outputs.geojsonDir, "lane_polygons.geojson"),
|
||||
network: path.join(area.outputs.geojsonDir, "network.json"),
|
||||
intersectionSurface: path.join(area.outputs.geojsonDir, "intersection_surface.geojson"),
|
||||
previewCss: path.join(path.resolve(__dirname, ".."), "lib", "cesium-preview.css"),
|
||||
previewJs: path.join(path.resolve(__dirname, ".."), "lib", "cesium-preview.js"),
|
||||
},
|
||||
outputs: {
|
||||
outputs: compressionComplete ? {
|
||||
vehicleRoute: area.outputs.vehicleRoute,
|
||||
vehicleModel: area.outputs.vehicleModel,
|
||||
} : {
|
||||
cesiumPreview: area.outputs.cesiumPreview,
|
||||
vehicleRoute: area.outputs.vehicleRoute,
|
||||
vehicleModel: area.outputs.vehicleModel,
|
||||
@@ -473,17 +476,13 @@ function stageManifestStatus(area, configPath = null) {
|
||||
},
|
||||
{
|
||||
stage: "compress",
|
||||
expected: fs.existsSync(area.outputs.compressedGlb),
|
||||
inputs: {
|
||||
expected: fs.existsSync(area.outputs.glb),
|
||||
inputs: {},
|
||||
outputs: {
|
||||
glb: area.outputs.glb,
|
||||
metadata: area.outputs.metadata,
|
||||
cesiumPreview: area.outputs.cesiumPreview,
|
||||
},
|
||||
outputs: {
|
||||
compressedGlb: area.outputs.compressedGlb,
|
||||
compressedMetadata: area.outputs.compressedMetadata,
|
||||
compressedCesiumPreview: area.outputs.compressedCesiumPreview,
|
||||
},
|
||||
},
|
||||
];
|
||||
return stages.map((entry) => {
|
||||
|
||||
@@ -6,11 +6,11 @@ const STAGES = [
|
||||
{ id: "blender", label: "Blender scene", description: "Generate the editable scene and render" },
|
||||
{ id: "cesium", label: "Cesium export", description: "Export GLB, metadata, and preview" },
|
||||
{ id: "preview", label: "Preview refresh", description: "Regenerate preview HTML and vehicles" },
|
||||
{ id: "compress", label: "Compressed preview", description: "Create optional compressed GLB artifacts" },
|
||||
{ id: "compress", label: "Compress delivery assets", description: "Replace standard GLB, metadata, and preview with compressed deliverables" },
|
||||
];
|
||||
|
||||
const ALIASES = {
|
||||
all: ["intermediates", "blender", "cesium"],
|
||||
all: ["intermediates", "blender", "cesium", "compress"],
|
||||
qgis: ["intermediates"], osm2streets: ["intermediates"], geojson: ["intermediates"], intermediate: ["intermediates"],
|
||||
intermediates: ["intermediates"], reimport: ["reimport"], gpkg: ["reimport"], blender: ["blender"], scene: ["blender"],
|
||||
cesium: ["cesium"], glb: ["cesium"], preview: ["preview"], html: ["preview"], cesiumPreview: ["preview"],
|
||||
|
||||
@@ -71,6 +71,10 @@ assert.equal(
|
||||
const configPath = path.join(tempDir, "area.json");
|
||||
fs.writeFileSync(configPath, `${JSON.stringify(base)}\n`);
|
||||
const area = normalizeAreaConfig(base);
|
||||
assert.equal(area.stages.compress, true);
|
||||
assert.equal("compressedGlb" in area.outputs, false);
|
||||
assert.equal("compressedMetadata" in area.outputs, false);
|
||||
assert.equal("compressedCesiumPreview" in area.outputs, false);
|
||||
fs.mkdirSync(area.outputs.geojsonDir, { recursive: true });
|
||||
for (const file of [area.outputs.glb, area.outputs.metadata, area.outputs.cesiumPreview, area.outputs.vehicleRoute, area.outputs.vehicleModel]) {
|
||||
fs.writeFileSync(file, "fixture\n");
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
const assert = require("assert");
|
||||
const { resolveStages, canonicalStages } = require("./lib/build-stages");
|
||||
assert.deepEqual(canonicalStages(resolveStages({}, ["compress", "blender", "preview"])), ["blender", "preview", "compress"]);
|
||||
assert.deepEqual(canonicalStages(resolveStages({}, ["all"])), ["intermediates", "blender", "cesium"]);
|
||||
assert.deepEqual(canonicalStages(resolveStages({}, ["all"])), ["intermediates", "blender", "cesium", "compress"]);
|
||||
assert.deepEqual(canonicalStages(resolveStages({ intermediates: true, blender: true, cesium: true, compress: true })), ["intermediates", "blender", "cesium", "compress"]);
|
||||
assert.throws(() => resolveStages({}, ["intermediates", "reimport"]), /mutually exclusive/);
|
||||
console.log("Build stage tests passed.");
|
||||
|
||||
Reference in New Issue
Block a user