Detect meaningful differences between web pages -- with Wayback Machine artifact cleaning, visual comparison, and significance scoring.
Comparing web pages sounds simple until you deal with Wayback Machine injection artifacts, insignificant whitespace noise, and visual regressions invisible to the DOM. Wayback-Diff is a purpose-built CLI that solves all three:
0 = no changes, 1 = low/medium, 2 = high).pip install wayback-diff
# Compare two pages
wayback-diff https://example.com/old https://example.com/new
# Compare a Wayback snapshot with the live site
wayback-diff https://web.archive.org/web/20230101/https://example.com/ https://example.com/
# Full report: visual diff + markdown
wayback-diff https://old.example.com https://new.example.com --visual --markdown
pip install wayback-diff
# With visual comparison support
pip install wayback-diff[visual]
git clone https://github.com/GeiserX/Wayback-Diff.git
cd Wayback-Diff
python3 -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
pip install -r requirements.txt
pip install -e .
For visual comparison support:
pip install -e ".[visual]"
docker build -t wayback-diff .
docker run --rm wayback-diff https://example.com/a https://example.com/b
wayback-diff https://example.com/page1 https://example.com/page2
The tool automatically detects Wayback Machine URLs and cleans injection artifacts before comparing:
# Archive vs. live site
wayback-diff https://web.archive.org/web/20230101/https://example.com/ https://example.com/
# Two archive snapshots
wayback-diff \
https://web.archive.org/web/20230101/https://example.com/ \
https://web.archive.org/web/20230601/https://example.com/
# Save to file
wayback-diff url1 url2 -o diff.txt
# JSON (for programmatic consumption)
wayback-diff url1 url2 --format json
# Unified diff
wayback-diff url1 url2 --format unified
# Crawl and compare across linked pages (depth-limited)
wayback-diff url1 url2 --traverse --depth 2
| Flag | Description |
|---|---|
--no-clean-wayback | Disable Wayback Machine artifact removal |
--no-ignore-whitespace | Treat whitespace changes as significant |
--timeout N | Set HTTP timeout in seconds (default: 30) |
--verbose | Enable detailed logging |
Take screenshots in one or more browsers and generate side-by-side difference images:
# Auto-detect all installed browsers
wayback-diff url1 url2 --visual
# Specific browsers
wayback-diff url1 url2 --visual --browsers chrome firefox edge opera
# Custom viewport
wayback-diff url1 url2 --visual --viewport-width 1280 --viewport-height 720
# Non-headless mode (for debugging)
wayback-diff url1 url2 --visual --no-headless
# Custom screenshot output
wayback-diff url1 url2 --visual --screenshot-dir ./my-screenshots
Visual comparison generates:
Generate comprehensive Markdown reports that include everything in a single reviewable document:
wayback-diff url1 url2 --visual --markdown --report-dir ./reports
Each report contains:
--visual is used)--traverse is used)Wayback-Diff returns meaningful exit codes designed for pipeline gates:
| Exit Code | Meaning |
|---|---|
0 | No differences detected |
1 | Low or medium significance changes |
2 | High significance changes detected |
name: Visual Regression Check
on:
pull_request:
jobs:
diff:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install Wayback-Diff
run: |
pip install -r requirements.txt
pip install -e ".[visual]"
- name: Compare staging vs production
run: |
wayback-diff \
https://staging.example.com \
https://production.example.com \
--visual --markdown --format json -o diff.json
- name: Upload report
if: always()
uses: actions/upload-artifact@v4
with:
name: diff-report
path: reports/
wayback-diff "$OLD_URL" "$NEW_URL" --format json -o result.json
EXIT_CODE=$?
if [ $EXIT_CODE -eq 2 ]; then
echo "BLOCKING: high-significance changes detected"
exit 1
elif [ $EXIT_CODE -eq 1 ]; then
echo "WARNING: minor changes detected"
fi
When a Wayback Machine URL is detected, the tool automatically:
web.archive.org/web/…/ prefixed URLs back to their originals.Every detected change is categorized:
| Level | Examples |
|---|---|
| High | Structural changes, content text, meta tags, scripts, stylesheets |
| Medium | Attribute changes, inline styling, div/span modifications |
| Low | Whitespace, comments, minor formatting |
The diff engine:
Summary statistics, significance breakdown, and detailed changes with context lines.
Structured output for programmatic processing:
{
"summary": {
"total_changes": 15,
"added": 5,
"removed": 3,
"modified": 7,
"high_significance": 2,
"medium_significance": 8,
"low_significance": 5
},
"changes": [
{
"type": "modified",
"old_text": "...",
"new_text": "...",
"significance": "high"
}
]
}
Standard unified diff format, compatible with patch and code review tools.
| Feature | Wayback-Diff | htmldiff | diff2html | BackstopJS | Percy |
|---|---|---|---|---|---|
| HTML-aware semantic diff | Yes | Yes | No | No | No |
| Wayback Machine artifact cleaning | Yes | No | No | No | No |
| Significance scoring | Yes | No | No | No | No |
| Visual (screenshot) comparison | Yes | No | No | Yes | Yes |
| Multi-browser support | Yes | N/A | N/A | Yes | Yes |
| Site-wide crawl and compare | Yes | No | No | Yes | No |
| Markdown report generation | Yes | No | No | No | No |
| CI/CD exit codes | Yes | No | No | Yes | Yes |
| Self-hosted / no SaaS | Yes | Yes | Yes | Yes | No |
| Free and open source | GPL-3.0 | MIT | MIT | MIT | Freemium |
pip install -r requirements-dev.txt
# Run tests
pytest tests/ -v
# Run with coverage
pytest tests/ -v --cov=wayback_diff --cov-report=html
Contributions are welcome. To get started:
git checkout -b feature/my-feature)pytest tests/ -v| Project | Description |
|---|---|
| Wayback-Archive | Download complete websites from the Wayback Machine with full asset preservation |
| Wayback-Diff | Intelligent web page comparison tool with Wayback Machine support |
| Way-CMS | Simple web CMS for editing HTML/CSS files downloaded from Wayback Archive |
| web-mirror | Mirror any webpage to a local server for offline access |
| media-download | Download all media files from any web page into a folder schema |
| n8n-nodes-way-cms | n8n community node for Way-CMS archived web content management |
This project is licensed under the GNU General Public License v3.0 (GPL-3.0). See the LICENSE file for details.
This software is not intended for commercial use.
Content type
Image
Digest
sha256:89fe452ac…
Size
72.3 MB
Last updated
8 days ago
docker pull drumsergio/wayback-diff:sha-3639636