Docker-based test runner with lua5.1, upstream LuaJIT, and Luau

.dockerignore

@@ -0,0 +1,15 @@
+node_modules
+web/node_modules
+web/dist
+web/dist-ssr
+web/public/docs
+web/test-results
+web/playwright-report
+.vite
+.vscode
+dist
+build
+Dockerfile
+.dockerignore
+.git
+.gitignore

.github/workflows/Test.yml

@@ -1,5 +1,5 @@
 name: Test
-on:  
+on:
   push:
     branches:
       - master
@@ -8,14 +8,10 @@ on:
       - master
 
 jobs:
-  test-linux:
+  test-docker:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repo
-        uses: actions/checkout@master
-      - name: Install Lua
-        uses: leafo/gh-actions-lua@master
-        with:
-          luaVersion: 5.1
-      - name: Run test case
-        run: lua ./tests.lua --Linux --CI
+        uses: actions/checkout@v4
+      - name: Build and run tests
+        run: bash scripts/run-tests.sh -b -n 10 --ci

.github/workflows/release.yml

@@ -14,12 +14,8 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - name: Install Lua 5.1
-        run: |
-          sudo apt-get update
-          sudo apt-get install -y lua5.1
-      - name: Run tests
-        run: lua5.1 ./tests.lua --Linux
+      - name: Build and run tests
+        run: bash scripts/run-tests.sh -b -n 10 --ci
 
   package:
     name: Package (${{ matrix.os_name }})

CONTRIBUTING.md

@@ -10,6 +10,29 @@ Thanks for contributing to Prometheus.
 - Add or update tests when changing behavior.
 - Document user-visible changes clearly.
 
+## Running Tests
+
+Tests run inside a Docker container with lua5.1 and Luau:
+
+```bash
+./scripts/run-tests.sh           # Run all tests (default: 10 iterations)
+./scripts/run-tests.sh -b        # Build image (needed first time or after Dockerfile changes)
+./scripts/run-tests.sh -n 5      # Run with 5 iterations
+./scripts/run-tests.sh -c config.lua  # Use a custom config
+./scripts/run-tests.sh -v        # Verbose output
+```
+
+### Test File Metadata
+
+Test files can include metadata comments at the top of the file:
+
+| Annotation | Effect |
+|-----------|--------|
+| `-- @skip` | Skip this test entirely |
+| `-- @luau-only` | Only run with Luau |
+| `-- @runtime lua51 luajit` | Only run with specified runtimes |
+| `-- @skip-preset Weak` | Skip a specific preset for this test |
+
 ## Reporting Bugs
 
 When opening a bug report, include:

Dockerfile

@@ -0,0 +1,23 @@
+FROM ubuntu:24.04
+
+RUN apt-get update && apt-get install -y \
+    lua5.1 \
+    git \
+    unzip \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+# Download pre-built Luau binary
+RUN curl -sSLo /tmp/luau-ubuntu.zip \
+    https://github.com/luau-lang/luau/releases/latest/download/luau-ubuntu.zip \
+    && unzip -j /tmp/luau-ubuntu.zip luau -d /usr/local/bin \
+    && chmod +x /usr/local/bin/luau \
+    && rm /tmp/luau-ubuntu.zip
+
+# Verify installations
+RUN lua5.1 -v && printf 'print("luau OK")\n' > /tmp/luau_check.lua && luau /tmp/luau_check.lua && rm /tmp/luau_check.lua
+
+WORKDIR /app
+COPY . /app
+
+ENTRYPOINT ["lua5.1", "docker-test-runner.lua"]

README.md

@@ -138,10 +138,14 @@ For more advanced use cases, configuration, and presets, see the [documentation]
 
 ## Tests
 
-To run the Prometheus test suite:
+The test suite runs inside Docker with lua5.1 and Luau:
 
 ```bash
-lua ./tests.lua [--Linux]
+./scripts/run-tests.sh           # Run all tests (default: 10 iterations)
+./scripts/run-tests.sh -b        # Build image and run tests
+./scripts/run-tests.sh -n 5      # Run with 5 iterations
+./scripts/run-tests.sh -c config.lua  # Use a custom config
+./scripts/run-tests.sh -v        # Verbose output
 ```
 
 ---