cgithub.git/src/scraper.ts, branch main

Read the pull request list from embedded JSON

2026-09-25T06:15:25Z

GitHub's pull request list no longer renders .js-issue-row rows server-side; the rows come from the repoPullsDashboardContentRoute payload instead. Co-Authored-By: Claude Opus 5.5

Show each repo nav item only if the GitHub page includes it

2026-09-25T06:15:02Z

Co-Authored-By: Claude Opus 5.5

Show issue and pull request status and labels in their lists

2026-09-25T05:43:53Z

Co-Authored-By: Claude Opus 5

Add issue details

2026-09-25T05:43:53Z

Co-Authored-By: Claude Opus 5

Add discussions

2026-09-25T05:43:53Z

Discussions are served from GitHub's older server-rendered HTML, with no embeddedData JSON to read, so the scraper walks the DOM the way the pull request and wiki code does. A discussion's timeline opens with its own post and continues with its comments, each carrying its replies nested inside it, which is where the recursion in _discussionComment.eta comes from. Categories, upvotes, and the answer a Q&A discussion is marked with all come along. Only what GitHub renders up front is shown: neither the next page of discussions nor the replies it hides behind a "Show N previous replies" button are fetched, so a comment says how many replies it is missing. Co-Authored-By: Claude Opus 5

Handle route errors in middleware

2026-09-04T03:28:52Z

Every page route wrapped its scrape in tryRender, which awaited the promise, rendered the template on success, and turned a failure into a redirect or an error page. Hono's onError does that mapping for all of them at once, so handlers now await their scrape inline and both wrappers are gone. Fragments under /api/ still fail as an empty body rather than a page; onError checks for them first, since a fragment turns a redirect into a 404 instead of following it. RedirectError becomes an abstract base with InternalRedirectError and ExternalRedirectError under it, so the handler dispatches on the type instead of sniffing the location for an https:// prefix. One behavior change: a thrown value that is not an Error no longer renders error.eta, because Hono only routes Error instances to onError. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01QERwoRJhuqMhKzzCUW4Skv

Render repository wikis

2026-09-03T05:09:10Z

Wikis are one of the last parts of GitHub still served as plain Rails HTML, with no embedded JSON to read, so this scrapes the markup: a page and any older revision of it, the _Sidebar and _Footer a wiki can define, and the page index. Every page is already in the sidebar of every page, since the "Show N more pages" button only unhides the rest, so linking the whole wiki costs no second request. Routes mirror GitHub's and resolve in its order, so an underscore name reserved for its own actions (_pages, _history, _new, _edit) can never be mistaken for a page; the ones we don't serve fall through to the usual redirect. A slug reaches us decoded and has to be encoded again rather than interpolated the way a ref is, or a page named C# would truncate at the fragment. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_013rQG1io1i4GJQW5RFcXNGd

Follow the commit history to its new home in the payload

2026-08-27T19:02:20Z

GitHub moved commitGroups from the top of the embedded JSON into a commitsRefRoute, alongside the code view's own routes, so every commits page threw. The groups themselves are unchanged. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BErPR92xppzGFXkPqXPYj2

Display PDF blobs

2026-08-27T18:53:38Z

GitHub renders a PDF with a viewer of its own and so ships no content for one, leaving the blob view with nothing to show. Its raw bytes come as application/octet-stream, which browsers download rather than display, so serve them from a new embed route that retypes them and point an at that. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BErPR92xppzGFXkPqXPYj2

Read the About box and latest release from JSON

2026-08-11T22:25:26Z

GitHub rebuilt the repository sidebar, so the selectors parseRepoInfo used matched nothing and the About box rendered empty. Everything but the latest release is in the page's embedded payload as sidebarAbout; the latest release is only in the _sidebar JSON already fetched for languages, so it arrives with that fragment and is moved beside the release count.