summaryrefslogtreecommitdiff
path: root/cgithub/src/scraper.test.ts
AgeCommit message (Collapse)Author
2026-08-27Display PDF blobsJoe Mou
GitHub renders a PDF with a viewer of its own and so ships no content for one, leaving the blob view with nothing to show. Its raw bytes come as application/octet-stream, which browsers download rather than display, so serve them from a new embed route that retypes them and point an <object> at that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BErPR92xppzGFXkPqXPYj2
2026-08-12Handle blobs GitHub cannot previewJoe Mou
Binary files with no preview (a .ttf, say) have no rawLines, no colorizedLines and no richText, so the blob view's final branch threw on a null textLines and the page 500ed. Offer the raw file instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11Read the About box and latest release from JSONJoe Mou
GitHub rebuilt the repository sidebar, so the selectors parseRepoInfo used matched nothing and the About box rendered empty. Everything but the latest release is in the page's embedded payload as sidebarAbout; the latest release is only in the _sidebar JSON already fetched for languages, so it arrives with that fragment and is moved beside the release count.
2026-08-11Add language percentages to repo pagesJoe Mou
GitHub no longer server-renders the repository sidebar; its React UI lazily fetches /:owner/:repo/_sidebar as JSON, which carries the language breakdown (name, percentage, color). getGitHubLanguages() uses that endpoint, and a new /api/:owner/:repo/languages fragment route renders the bar and legend the same way refs and latest-commit already work, so repo page loads still cost a single scrape. The latest-commit placeholder loader in refs.js is generalized over .placeholder[data-src] so both placeholders share it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-22Add async branch/tag dropdown and latest-commit barJoe Mou
Adds two lazily-fetched UI pieces to match vanilla GitHub while keeping initial page loads plain server-rendered HTML: a branch/tag selector (native <dialog>, CSS anchor-positioned to its button, light-dismissed via closedby="any") and a latest-commit bar above the file listing. Both are populated by new /api/:owner/:repo/refs/:type and /api/:owner/:repo/latest-commit/:branch/:path routes; the latter calls GitHub's own /latest-commit JSON endpoint (the same one the React UI uses) rather than scraping the full commit history page.
2026-07-15Replace null overview files with HTML commentsJoe Mou
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15Owner page nav menu and layout restructureJoe Mou
Add a menu bar of profile tab links, split the org repository list from pinned repos, and restructure the owner template to match repo pages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15Add organization support to owner pagesJoe Mou
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15Remove watchers, no longer rendered by GitHubJoe Mou
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15User profile pagesJoe Mou
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-05-27Pull request indexJoe Mou
2026-05-27Add repo-scoped search support for issues and PRsJoe Mou
We do not support hybrid issue/PR search. Any searches on the issues endpoint will redirect to an explicit issue-only search.
2026-05-27Update unit testsJoe Mou
2026-04-28Support redirectsJoe Mou
2026-04-20Apply overlapping styling drectives instead of duplicating contentJoe Mou
2026-04-20Update for unexpected actions/deploy-pages releaseJoe Mou
2026-03-14Fix tree overviewHtml: use tree.readme instead of overview.overviewFilesJoe Mou
2026-03-13Fix GitHub scraping for new JSON structure, update TypeScript interfacesJoe Mou
GitHub restructured their embedded JSON: tree/blob data is now split across codeViewRepoRoute, codeViewTreeRoute, codeViewBlobLayoutRoute, codeViewBlobLayoutRoute.StyledBlob, codeViewBlobRoute, and the shared codeViewLayoutRoute (repo, refInfo, path). Replace RawPayload and OverviewFile with typed interfaces for each route, add AppPayload as the top-level union, and simplify parsePayload to always extract react-app.embeddedData.
2026-02-21Release assetsJoe Mou
Also fix release content parsing.
2026-02-21Add latest release detailsJoe Mou
By Gemini
2026-02-21Add single release detail pages and refactor templatesJoe Mou
Refactored releases display into reusable partial template (_release.eta) and added support for viewing individual releases at /:owner/:repo/releases/tag/:tag with navigation back to all releases. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-21Parse release page HTMLJoe Mou
Fixes reading complex release descriptions that had unbalanced tags. Written with Gemini.
2026-02-11Add GitHub releases supportJoe Mou
Scrapes releases page HTML to display version tags, publish dates, and changelog content. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-28Commits listJoe Mou
2026-01-28Image supportJoe Mou
2026-01-28Syntax highlightingJoe Mou
Sometimes GitHub returns colorizedLines and sometimes it returns a terser stylingDirectives, which needs to be applied to rawLines.
2026-01-28Render Markdown blobsJoe Mou
2026-01-27List issuesJoe Mou
2026-01-27Propagate GitHub HTTP errorsJoe Mou
Render `error.eta` on HTTP error from GitHub. Implemented with Gemini.
2026-01-27GitHub scraper testJoe Mou