-
Notifications
You must be signed in to change notification settings - Fork 25
[WJ-1293] Add wws service to support wjfiles routes, add Caddy for reverse proxying and HTTPS #2283
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We're not handling missing sites at this level.
Host information is already passed through.
For the build and running tests.
This logic is now handled by Caddy when it reads our Caddyfile. No need for this to be done at runtime by anyone.
For instance, if my files domain is wjfiles.com, I can't make a custom domain for my site which is "foo.awesomesite.wjfiles.com". This resolves a TODO.
c331f7d to
7e5a81d
Compare
Since we moved the admin panel route, we can also resolve the issue with _admin being "not a page" for the purposes of stuff like orange links by actually just making a page-based redirect! See WJ-331.
This was accidentally added here.
7e5a81d to
5b5d0fc
Compare
5b5d0fc to
0500f57
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Apologies in advance for the size for this PR. Sorry it's been two months!
As seems to be common in this project, attempting to introduce a significant piece of infrastructure / a new abstraction (in this case wjfiles) revealed several core issues which I've had to work through. Ultimately, this PR changed directions several times as new issues were revealed, but in the end I think the infra is in a better state now for having found these problems.
Major Changes
Caddyfileconfiguration file:CaddyServiceto DEEPWELL. This new service is responsible for generating theCaddyfileconfig that is fed to caddy. Instead of doing a host lookup every time we get a new request, we just generate the configuration with all the relevant matching (e.g. this custom domain refers to this site, this site is preferred and should be redirected to, etc.) This is responsible for adding internal HTTP headers,X-Wikijump-Site-IdandX-Wikijump-Site-Slug, which terminates the host logic and allows wws and framerail to easily lookup site information using the site ID. There are tests to ensure that the config generation is as expected for various options.api→deepwellweb→framerailwwscaddyERROR XF-1003. This way developers can be alerted to the matter and shown an indication of where the problem may be despite the other limitations that clearly existed. (If you're curious,ERRORis obvious and hopefully intelligible English for most people given our Anglocentric modern web, andXFis a combination ofX, which seems like a thing you'd see in an error code, andFfor fallback. The number should increase monotonically so we can uniquely identify issues.)All the other changes should fall into one of the above categories, or be one of many miscellenous fixes or updates.
Notes and Caveats
wwsis not finished, but I figured future PRs would be easier to review if they were not so bulky. I have the "happy path" for file requests finished, in addition to simpler handlers, but code and html requests are entirely stubbed and will be done separately in WJ-1224.deploy_on_pushsettings.Caddyfileinstead of the underlying JSON for two reasons. First,Caddyfiles are human-readable and much easier to inspect for correctness visually. (This is why I have unit tests forCaddyServicegeneration.) Second, I am not nearly as familiar with how to properly generate those JSON rules, and I don't see any issue with caddy generating it for us.Caddyfileconfiguration from DEEPWELL on boot, and a cronjob exists inside to auto-update it every hour. This is my solution to having site host changes (like adding a custom domain) become present in the infrastructure, though it does add a time delay (since I'm not sure how to securely make the updates push-only) and that a bug inCaddyServicemeans all site updates will be frozen (due to Caddy's auto-rollback mechanism) until it's fixed. This regular config update should have no effect on the infrastructure: according to Caddy's docs, the reload is "lightweight, efficient, and incur[s] zero downtime", and the change isn't even applied anyways if the configuration is the same (which it should always be unless somebody adds a new site or changes one of its domain settings)./-/health-checkroute pings DEEPWELL, which in turn pings Postgres and Redis. It's a more "proper" health check. There is also/-/health-check/caddy, which is simply that Caddy is able to respond. The latter is for debugging issues with routing, so you can determine that things are working as far as Caddy, but where reverse proxying further is broken.X-Wikijump-Target-Serverheader can be eithermainorfiles, and is used in cases where a request is hitting the main server (e.g.wikijump.com), but wws is handling the request. An example would berobots.txt, which could differ betweenfoo.wikijump.comandfoo.wjfiles.com. Planning for how exactly sites can customizerobots.txt's contents has yet to be planned, but the groundwork for it is laid here, since we can add appropriate code to wws for that.