Commit graph

1304 commits

Author SHA1 Message Date
4c4bd46aa8 fup: factor out metadata retrieval into a separate file
We'll need this for cleanup operations as well. This should likely be factored
out again into an entirely separate package that deals with storage access.
2021-03-21 15:17:46 +00:00
7bac0aee74 fup: implement deleting expired files on request
This is a very basic implementation of gating that we only return files which
haven't expired.  If the file has expired, then we just delete it.
2021-03-21 14:56:42 +00:00
0e69199569 fup: shore up serving and add tests for supported methods
We support several methods of serving assets:

* Redirect to blobstore
    - This requires the backend to support signed URLs.
      We rely on the backend to support HTTP semantics, like supporting
      Range headers.
* Serve-using-http.ServeContent
    - This requires the backend to actually be providing a io.ReadSeeker.
      net/http provides Range/If- conditional support.
* Serve-proxy
    - This is the safest and most compatible method.
      We don't support conditionals nor Range headers. This mode is unlikely
      to be suitable for multimedia, like MP3s or video.
2021-03-21 14:35:04 +00:00
04eb6c0299 fup: fixup 2021-03-21 03:29:34 +00:00
08098fb666 fup: add file serving
This adds both redirect-to-signed-URL and proxy fileserving.

The proxy fileserving is somewhat limited: we don't support the Range header,
and it isn't easy to reuse the net/http ServeContent implementation because
that requires a SeekCloser. I think it might be possible to "bodge" a
SeekCloser on top of dynamically opening files, but it'll be a bit wonky and
will be slower than strictly necessary.
2021-03-21 03:04:38 +00:00
e21db7a061 fup: add a template function for getting paths to static assets
We add the hash of the file to the static assets, so they can be cached
indefinitely.  This also, however, means that we need some way of referring to
them.
2021-03-21 03:03:15 +00:00
5846385513 fup: move hashfs into its own package.
We need to depend on its API for fuphttp, so it's better if it's a separate
package to avoid embedding things we don't need.

In general it's probably a good idea to separate the logic from the embedded
content...
2021-03-20 23:43:59 +00:00
25443cfaab fup: add bucket-url flag for specifying storage
This allows setting the persistent storage location to save files into.

NOTE: defaults to mem:// for development purposes, which... is gonna be a bad
time.
2021-03-20 23:33:39 +00:00
7e4fb52cef fup: add viper for persistent config 2021-03-20 23:32:56 +00:00
cb44e7894f fup: add gocloud.dev blob backends 2021-03-20 23:17:02 +00:00
Default email
71ea7b1625 Project import generated by Copybara.
GitOrigin-RevId: f5e8bdd07d1afaabf6b37afc5497b1e498b8046f
2021-03-19 18:17:44 +01:00
a0a853c3c9 3p/nixpkgs: fix python38Packages.cherrypy 2021-03-21 03:20:06 +00:00
Default email
07d6a74cbb Project import generated by Copybara.
GitOrigin-RevId: f5e8bdd07d1afaabf6b37afc5497b1e498b8046f
2021-03-19 18:17:44 +01:00
787b04737e treewide: add some SPDX headers 2021-03-20 20:46:56 +00:00
a261baedf4 fup: add SPDX headers 2021-03-20 20:40:40 +00:00
bcd39fae10 fup: create serve subcommand and fuphttp package
This is the skeleton of the application. Let's goooo!
2021-03-20 19:53:43 +00:00
a1bda601a9 fup: add extractstatic command
This extracts the hash-named static assets into a directory.  This is primarily
useful if you have multiple instances of fup running and you therefore need to
serve /static from a shared filesystem of some sort.
2021-03-20 19:53:04 +00:00
38d9449777 fupstatic: init embedding package 2021-03-20 19:52:10 +00:00
2104261536 fupstatic: add some basic templates/css/js. 2021-03-20 19:51:57 +00:00
03cf4ea939 fupstatic: create hashfs
I want to serve static files with hashes in, and I'm too lazy
to construct a preprocessing step.
2021-03-20 19:51:32 +00:00
ecbf5a6450 fup: init 2021-03-20 19:49:33 +00:00
35cc195717 common: remove everything from hosts files 2021-03-20 16:42:08 +00:00
99dce2de2a as205479.net: add totoro.int 2021-03-20 16:41:26 +00:00
33fd1da091 dns: add blades to zone 2021-03-20 15:22:09 +00:00
4c78164384 ops/nixos/common: set search domains 2021-03-20 15:01:28 +00:00
0d9d3b65be gitlab-ci: run dockerpush on clouvider-lon01 instead 2021-03-20 13:46:39 +00:00
5cf89fbc2f switch-prebuilt: check for existence before nix copy 2021-03-20 13:37:08 +00:00
422c47c3e0 switch-prebuilt: run stuff assuming we're a trusted-user 2021-03-20 13:22:17 +00:00
e1aa4132d8 hack/deploy: we have jq now 2021-03-20 13:11:23 +00:00
c7a66f35e6 hack/deploy: move ssh_cmd to right place 2021-03-20 13:06:52 +00:00
ed1c39a5f1 hack/deploy: use switch-prebuilt 2021-03-20 13:03:07 +00:00
be5eee48b3 switch-prebuilt: init 2021-03-20 12:39:23 +00:00
154db9706a lib/common: add deployer to trustedUsers 2021-03-20 12:34:01 +00:00
7737f962a9 marukuru/deployer: add jq somewhere more sensible 2021-03-20 12:17:41 +00:00
d8086e7042 ops/nixos: add jq everywhere 2021-03-20 12:11:45 +00:00
705bcd9446 marukuru/deployer: add jq 2021-03-20 11:57:21 +00:00
b3df707421 gitlab-ci: try to see if we can parse out the system 2021-03-20 03:12:05 +00:00
713c2e755b gitlab-ci: add systems.json to pipeline artifacts 2021-03-20 02:36:59 +00:00
627c8bf17c lib/coredns: fix firewall 2021-03-20 02:06:08 +00:00
b0a6ebe52d ops/nixos: add coredns 2021-03-20 02:03:23 +00:00
7f27f9fb79 tuvok/paris: enable keepalived 2021-03-20 01:08:33 +00:00
c51e5d478d lib/common: add --delete-older-than 2021-03-19 21:29:54 +00:00
9036f02fd0 blade-tuvok: allow 80/443 2021-03-19 21:27:42 +00:00
9ddb5d75f2 blade: restrict ceph firewall rules to storage network 2021-03-19 21:27:15 +00:00
4f5f2a780a nixos-mgmt: init 2021-03-19 20:28:24 +00:00
3f3c92addc blade-tuvok: serve objdump directly 2021-03-19 19:45:03 +00:00
82db96db25 gitlab-ci: add {lux,las,nyc}01-frantech 2021-03-19 16:17:14 +00:00
10c7ee76b3 {las,lux,nyc}01-frantech: add tailscale IPs 2021-03-19 16:16:06 +00:00
a7657b7789 ci-root: remove the CD; it's huge and unnecessary most of the time 2021-03-19 16:14:48 +00:00
0f0f61f2df installcd: add to ci-root; tweak store 2021-03-19 01:19:14 +00:00