Vendor the H5P core from h5p-php-library and retire h5p-standalone

.github/codeql/codeql-config.yml

@@ -0,0 +1,7 @@
+name: CodeQL config
+
+# The H5P core as h5p-php-library has it, at the commit vendor/h5p-core/SOURCE.json names, and
+# not changed here: a finding in it is upstream's, reported there and taken with `pnpm core:sync`.
+# Until #37 the same code came from h5p-standalone under node_modules/, which was not scanned either.
+paths-ignore:
+  - packages/runtime/vendor

.github/workflows/codeql.yml

@@ -25,4 +25,5 @@ jobs:
         with:
           languages: javascript-typescript
           build-mode: none
+          config-file: ./.github/codeql/codeql-config.yml
       - uses: github/codeql-action/analyze@v4

README.md

@@ -270,9 +270,9 @@ The player is MIT ([LICENSE](https://github.com/missing-elements/h5p-offline-pla
 Its Service Worker bundles [zip.js](https://github.com/gildas-lormeau/zip.js), BSD-3-Clause
 ([NOTICE.md](https://github.com/missing-elements/h5p-offline-player/blob/main/packages/player/NOTICE.md)).
 
-The H5P runtime the frame loads is GPL-3.0, from [h5p-php-library](https://github.com/h5p/h5p-php-library)
-by way of [h5p-standalone](https://github.com/tunapanda/h5p-standalone)
-([#188](https://github.com/tunapanda/h5p-standalone/issues/188)). It is published separately as
+The H5P runtime the frame loads is GPL-3.0: the H5P core from
+[h5p-php-library](https://github.com/h5p/h5p-php-library), at a commit its notice names, and the
+script that boots it. It is published separately as
 [`@missing-elements/h5p-runtime`](https://www.npmjs.com/package/@missing-elements/h5p-runtime);
 the player talks to it only over HTTP and `postMessage`. Its
 [NOTICE.md](https://github.com/missing-elements/h5p-offline-player/blob/main/packages/runtime/NOTICE.md)

docs/h5p-standalone.md

@@ -1,6 +1,8 @@
 # Coming from h5p-standalone
 
-The element runs h5p-standalone inside its frame and takes its options by the same names where they still apply.
+The element takes h5p-standalone's options by the same names where they still apply. It ran
+h5p-standalone inside its frame until the runtime vendored the H5P core from h5p-php-library and
+loaded it itself; the libraries load in the same order, and the options mean what they meant.
 
 | h5p-standalone | Here | Note |
 |---|---|---|

package.json

@@ -22,6 +22,7 @@
     "libraries": "pnpm --filter @missing-elements/h5p-demo libraries",
     "sweep": "pnpm --filter @missing-elements/h5p-demo sweep",
     "l10n": "pnpm --filter @missing-elements/h5p-runtime l10n",
+    "core:sync": "pnpm --filter @missing-elements/h5p-runtime core:sync",
     "cmi5:catapult": "pnpm --filter @missing-elements/h5p-demo cmi5:catapult"
   },
   "engines": {

packages/embed/tests/build.test.ts

@@ -87,7 +87,7 @@ describe('the site', () => {
     for (const name of ['index.html', 'main.js', 'embed.js', 'embed.css', 'resizer.js', 'config.js', 'h5p-player.js', 'h5p-sw.js', 'h5p-jobs.js', 'libraries/index.json', 'libraries.txt', 'NOTICE.txt', '_headers', 'vercel.json', 'Caddyfile', 'nginx.conf', '.htaccess']) {
       expect(existsSync(join(out, name)), name).toBe(true)
     }
-    for (const name of ['frame-boot.js', 'frame.bundle.js', 'main.bundle.js', 'h5p.css', 'LICENSE.txt', 'NOTICE.txt']) {
+    for (const name of ['frame-boot.js', 'frame.bundle.js', 'h5p.css', 'LICENSE.txt', 'NOTICE.txt']) {
       expect(existsSync(join(out, 'frame-assets', name)), name).toBe(true)
     }
     // The pack's parts, every one its index names, for the player to fetch as a package needs them.

packages/player/NOTICE.md

@@ -47,8 +47,9 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 ## The H5P runtime is not in this package
 
 The element plays content inside a frame that runs the H5P core runtime, and that runtime is
-**GPL-3.0** — it comes from [h5p/h5p-php-library](https://github.com/h5p/h5p-php-library) by way
-of [h5p-standalone](https://github.com/tunapanda/h5p-standalone). Up to 0.4 this package shipped
+**GPL-3.0** — it is the core of [h5p/h5p-php-library](https://github.com/h5p/h5p-php-library),
+which came by way of [h5p-standalone](https://github.com/tunapanda/h5p-standalone) until the
+runtime vendored it from upstream at a commit. Up to 0.4 this package shipped
 it under `dist/frame-assets/` and was published as `(MIT AND GPL-3.0-only)`. From 0.5 it does
 not: the runtime, its stylesheet and fonts, and the script that boots it inside the frame are
 [`@missing-elements/h5p-runtime`](https://www.npmjs.com/package/@missing-elements/h5p-runtime),

packages/player/scripts/build-fixtures.mjs

@@ -473,8 +473,8 @@ for (const [name, minor, content] of [
   ])
 }
 
-// 7. Library folders with no version suffix, the way older packages are built. h5p-standalone
-//    probes the versioned name first, takes the 404 and falls back to the bare one.
+// 7. Library folders with no version suffix, the way older packages are built. Each library is
+//    looked for under the versioned name first and under the bare one after the 404.
 built.push([
   'unversioned.h5p',
   await writeArchive('unversioned.h5p', async (writer) => {
@@ -485,6 +485,34 @@ built.push([
   })
 ])
 
+// 7a. The same, carrying the extra library too, which h5p.json lists after the content type. Played
+//     with the pack, its content type plays at the pack's 1.2, from a folder named with its
+//     version, and the extra library, which the pack has at the same version, from the package's
+//     own folder named without one. h5p-standalone named every folder one way, by whether the first
+//     dependency answered under its versioned name, so the two kinds of folder side by side ended
+//     its boot on the extra library's 404 (#37).
+const UNVERSIONED_MANIFEST = { ...JSON.parse(await readFile(resolve(sourceDir, 'h5p.json'), 'utf8')) }
+UNVERSIONED_MANIFEST.preloadedDependencies = [
+  ...UNVERSIONED_MANIFEST.preloadedDependencies,
+  { machineName: 'H5P.OfflineExtra', majorVersion: 1, minorVersion: 0 }
+]
+built.push([
+  'unversioned-pack.h5p',
+  await writeArchive('unversioned-pack.h5p', async (writer) => {
+    for (const { entryName, path } of baseFiles) {
+      if (entryName === 'h5p.json') {
+        await writer.add('h5p.json', new TextReader(JSON.stringify(UNVERSIONED_MANIFEST)))
+        continue
+      }
+      const bare = entryName.replace(/^H5P\.OfflineTest-1\.0\//, 'H5P.OfflineTest/')
+      await writer.add(bare, new Uint8ArrayReader(new Uint8Array(await readFile(path))))
+    }
+    for (const { entryName, path } of extraLibFiles) {
+      await writer.add(entryName.replace(/^H5P\.OfflineExtra-1\.0\//, 'H5P.OfflineExtra/'), new Uint8ArrayReader(new Uint8Array(await readFile(path))))
+    }
+  })
+])
+
 // 8. A zip that is not an H5P package at all.
 built.push([
   'not-h5p.h5p',

packages/player/scripts/lib/fixture-names.mjs

@@ -29,6 +29,7 @@ export const GENERATED_FIXTURES = [
   'upgrade-1.2.h5p',
   'upgrade-fails.h5p',
   'unversioned.h5p',
+  'unversioned-pack.h5p',
   'corrupt.h5p',
   'html-page.h5p'
 ]

packages/player/scripts/sync-h5p-assets.mjs

@@ -40,7 +40,7 @@ await cp(RUNTIME_DIST, targetDir, { recursive: true })
 // Written only when it changes, like the version stamp below, so a sync leaves git clean.
 const fontsModulePath = resolve(rootDir, 'src', 'frame-fonts.ts')
 const fontsModule = `// Written by scripts/sync-h5p-assets.mjs from @missing-elements/h5p-runtime ${manifest.runtime}'s
-// MANIFEST.json (h5p-standalone ${manifest.h5pStandalone}). Do not edit; run \`pnpm sync:assets\` in packages/player.
+// MANIFEST.json (the H5P core at ${manifest.core.commit.slice(0, 10)}). Do not edit; run \`pnpm sync:assets\` in packages/player.
 
 /**
  * The text faces the runtime's stylesheet declares, in its order, by the file names under its
@@ -97,5 +97,5 @@ const currentPackIndex = (await exists(packIndexPath)) ? await readFile(packInde
 if (currentPackIndex !== packIndexModule) await writeFile(packIndexPath, packIndexModule, 'utf8')
 
 console.log(
-  `[sync-h5p] @missing-elements/h5p-runtime ${manifest.runtime} (h5p-standalone v${manifest.h5pStandalone}) -> public/frame-assets`
+  `[sync-h5p] @missing-elements/h5p-runtime ${manifest.runtime} (the H5P core at ${manifest.core.commit.slice(0, 10)}, core API ${manifest.core.coreApi}) -> public/frame-assets`
 )

packages/player/src/frame-fonts.ts

@@ -1,5 +1,5 @@
 // Written by scripts/sync-h5p-assets.mjs from @missing-elements/h5p-runtime 0.2.5's
-// MANIFEST.json (h5p-standalone 3.8.2). Do not edit; run `pnpm sync:assets` in packages/player.
+// MANIFEST.json (the H5P core at cb64a1f388). Do not edit; run `pnpm sync:assets` in packages/player.
 
 /**
  * The text faces the runtime's stylesheet declares, in its order, by the file names under its

packages/player/src/h5p-offline-player.ts

@@ -64,7 +64,6 @@ const DEV_ASSETS_BASE = '/frame-assets/'
 
 /** File names inside the runtime's directory, relative to the assets base. */
 const ASSET_FILES = {
-  mainJs: 'main.bundle.js',
   frameJs: 'frame.bundle.js',
   frameCss: 'h5p.css',
   bootJs: 'frame-boot.js',
@@ -122,7 +121,6 @@ const BLOB_JOBS_SCRIPT: JobsScript = {
 /** The runtime inside a directory: the dev server's, the default beside the element, or an `assets-base`. */
 function assetsIn(base: string): FrameAssets {
   return {
-    mainJs: new URL(ASSET_FILES.mainJs, base).href,
     frameJs: new URL(ASSET_FILES.frameJs, base).href,
     frameCss: new URL(ASSET_FILES.frameCss, base).href,
     bootJs: new URL(ASSET_FILES.bootJs, base).href,
@@ -1241,9 +1239,12 @@ export class H5PPlayerElement extends HTMLElement {
   }
 
   set runtime(value: FrameAssets | null) {
-    this.runtimeValue = value && [value.mainJs, value.frameJs, value.frameCss, value.bootJs].every((url) => typeof url === 'string' && url)
+    // A runtime up to 0.2 names h5p-standalone's loader as well, `mainJs`, and is kept with it:
+    // the worker still writes the frame document such a runtime's boot script expects.
+    const files = value && [value.frameJs, value.frameCss, value.bootJs]
+    this.runtimeValue = files && files.every((url) => typeof url === 'string' && url)
       ? {
-          mainJs: value.mainJs,
+          ...(typeof value.mainJs === 'string' && value.mainJs ? { mainJs: value.mainJs } : {}),
           frameJs: value.frameJs,
           frameCss: value.frameCss,
           bootJs: value.bootJs,

packages/player/src/shared/protocol.ts

@@ -245,11 +245,16 @@ export interface LoadSettings {
  * is GPL-3.0 and the player is MIT; the player carries none of these files and only names them.
  */
 export interface FrameAssets {
-  /** h5p-standalone `main.bundle.js` — the loader that walks dependencies and boots the runtime. */
-  mainJs: string
-  /** h5p-standalone `frame.bundle.js` — h5p.js, jQuery and the core runtime. */
+  /**
+   * h5p-standalone's `main.bundle.js`, the loader of the runtime up to 0.2, which walked the
+   * libraries in the frame and booted the core: present in a record an element wrote with such a
+   * runtime, whose frame document the worker still writes the way it expects. A later runtime has
+   * none; its boot script loads what the worker's `boot` route names (`LibraryLoad`).
+   */
+  mainJs?: string
+  /** `frame.bundle.js` — the H5P core's scripts, jQuery and h5p.js among them. */
   frameJs: string
-  /** h5p-standalone `styles/h5p.css`, rebuilt to stand alone: icon fonts and images inlined. */
+  /** `h5p.css`, the core's stylesheets rebuilt to stand alone: icon fonts and images inlined. */
   frameCss: string
   /**
    * The frame's boot script, `frame-boot.js`, which the worker names in the frame document.
@@ -278,6 +283,28 @@ export interface FrameFont {
   url: string
 }
 
+/**
+ * What a frame loads, as the worker's `boot` route answers it (`sw/library-load.ts`): the boot
+ * script of a runtime after 0.2 asks for it and builds `H5PIntegration` from it, where h5p-standalone
+ * walked the libraries itself. A contract between the player and the runtime, like the boot
+ * configuration: a field the boot script needs is one every worker it pairs with writes.
+ */
+export interface LibraryLoad {
+  /** The main library at the version served, `H5P.QuestionSet 1.21`: `H5PIntegration`'s `library`. */
+  library: string
+  /** Every library's scripts, dependencies first, as entry names under the package's root. */
+  scripts: string[]
+  /** Every library's stylesheets, in the same order. */
+  styles: string[]
+  /**
+   * The core's `libraryDirectories`: for a library served from a folder its version does not name,
+   * `H5P.Foo-1.2` → `H5P.Foo`, so that `H5P.getLibraryPath` finds it.
+   */
+  directories: Record<string, string>
+  /** The library folders, reachable from the package, that carry an `upgrades.js`. */
+  upgrades: string[]
+}
+
 export type ErrorCode =
   | 'no-cors'
   | 'no-worker'

packages/player/src/sw/frame-document.ts

@@ -88,6 +88,8 @@ export interface FrameDocumentOptions {
   pkgId: string
   /** Absolute URL of the package root on the virtual file server, without a trailing slash. */
   virtualRoot: string
+  /** Absolute URL of the load's `boot` route, where a runtime after 0.2 asks what to load. */
+  bootUrl: string
   assets: FrameAssets
   nonce: string
   title?: string
@@ -102,7 +104,8 @@ export interface FrameDocumentOptions {
   frameOptions?: FrameOptions
   /**
    * The library folders that carry an `upgrades.js`, for the boot script's upgrade of content
-   * written for an older minor than the one served.
+   * written for an older minor than the one served: a runtime up to 0.2 reads it here, a later
+   * one from the `boot` route.
    */
   upgrades?: string[]
   /** The language tag the element was in, which H5P's own strings follow; English without one. */
@@ -132,7 +135,7 @@ function escapeHtml(value: string): string {
 export function bootScriptUrl(assets: FrameAssets): string {
   if (assets.bootJs) return assets.bootJs
   try {
-    return new URL('frame-boot.js', assets.mainJs).href
+    return new URL('frame-boot.js', assets.mainJs ?? '').href
   } catch {
     // A `mainJs` that is not a URL has no sibling to resolve; the bare name is left to the frame's
     // own base, as a record from before 0.5 would have resolved it.
@@ -176,7 +179,7 @@ function assetOrigins(assets: FrameAssets, base: string, language: string): stri
   const origins = new Set<string>()
   const strings = l10nScriptUrl(assets, language)
   const urls = [
-    assets.mainJs,
+    ...(assets.mainJs ? [assets.mainJs] : []),
     assets.frameJs,
     assets.frameCss,
     bootScriptUrl(assets),
@@ -279,7 +282,7 @@ function sanitizeFrameOptions(options: FrameOptions | undefined): FrameOptions |
 function untrustedScriptSources(options: FrameDocumentOptions): string[] {
   const strings = l10nScriptUrl(options.assets, frameLanguage(options.language))
   const files = [
-    options.assets.mainJs,
+    ...(options.assets.mainJs ? [options.assets.mainJs] : []),
     options.assets.frameJs,
     bootScriptUrl(options.assets),
     ...(strings ? [strings] : []),
@@ -368,19 +371,25 @@ export function buildFrameDocument(options: FrameDocumentOptions): string {
   const language = frameLanguage(options.language)
   const l10nJs = l10nScriptUrl(assets, language)
 
+  // A runtime up to 0.2 boots through h5p-standalone's `main.bundle.js`, which the document loads
+  // ahead of the boot script, and takes the upgrade list from here; a later one has no such file
+  // and asks the `boot` route for what to load.
+  const loader = assets.mainJs
+
   // A JSON block, not a script: it is data, so it needs no nonce, and the boot script proper —
   // `frame-boot.js` from the runtime package, loaded by URL under the nonce — is the same bytes
   // for every package. `<` is escaped so no value can end the block early.
   const bootConfig = JSON.stringify({
     pkgId,
     h5pJsonPath: virtualRoot,
+    boot: loader ? undefined : options.bootUrl,
     frameJs: assets.frameJs,
     frameCss: assets.frameCss,
     title: options.title,
     // Checked again on the way out, as the options are: the record came back from IndexedDB.
     metadata: contentMetadata(options.metadata),
     options: sanitizeFrameOptions(options.frameOptions),
-    upgrades: options.upgrades,
+    upgrades: loader ? options.upgrades : undefined,
     // What the boot script picks H5P's own strings by, when it has the runtime's translations.
     language: l10nJs ? language : undefined,
     // The boot script checks the content against its libraries' semantics, and asks for it with
@@ -390,9 +399,10 @@ export function buildFrameDocument(options: FrameDocumentOptions): string {
 
   const fontFaces = fontFaceRules(assets.fonts)
   // The core's strings in every language the runtime has, a script that leaves them on `window`
-  // for the boot script, which picks the element's language and hands its strings to
-  // h5p-standalone: so it comes before it.
+  // for the boot script, which picks the element's language and builds the core's settings with
+  // its strings: so it comes before it.
   const strings = l10nJs ? `<script nonce="${nonce}" src="${escapeHtml(l10nJs)}" charset="UTF-8"></script>\n` : ''
+  const legacyLoader = loader ? `<script nonce="${nonce}" src="${escapeHtml(loader)}" charset="UTF-8"></script>\n` : ''
 
   // The last rule in the style block is for H5P.Video's YouTube handler, which pins its iframe
   // over the 16:9 box it builds with one line that reaches into the YouTube API object's minified
@@ -416,8 +426,7 @@ ${fontFaces}
 </head>
 <body>
 <div id="h5p-root"></div>
-<script nonce="${nonce}" src="${escapeHtml(assets.mainJs)}" charset="UTF-8"></script>
-${strings}<script type="application/json" id="h5p-boot-config">${bootConfig}</script>
+${legacyLoader}${strings}<script type="application/json" id="h5p-boot-config">${bootConfig}</script>
 <script nonce="${nonce}" src="${escapeHtml(bootScriptUrl(assets))}"></script>
 </body>
 </html>`