Compare commits

...

24 Commits

Author SHA1 Message Date
Asaf Gartner 02d51a8bfe Merge branch 'streams_update' 2022-05-30 17:33:52 +03:00
Asaf Gartner 0d0773fd0e Reverse streamer sort and replaced green circle with red 2022-05-27 16:34:34 +03:00
Asaf Gartner 09c4561428 Slightly improved discord message formatting. 2022-05-27 11:55:57 +03:00
Asaf Gartner 8495982d3f Added persistent vars and improved stream tracking on discord. 2022-05-27 11:37:43 +03:00
Ben Visness 68a00c91db Rename the email override field for clarity 2022-05-25 17:39:57 -05:00
bvisness 2f19e6e1b8 Add note about Go version to the readme 2022-05-25 22:33:28 +00:00
Ben Visness 475716322f Fix up missing avatars 2022-05-21 16:44:39 -05:00
Ben Visness 6dd11aa970 Delete resetdb.sh 2022-05-16 10:26:56 -05:00
Ben Visness 491ae57590 Tweak Postgres help message on failed seed 2022-05-14 23:06:48 -05:00
Ben Visness c1fa6cae13 Integrate Nick's local S3 server
Works like a charm!

Small tweak for clarity
2022-05-14 00:48:19 -05:00
bvisness 4187a3b6ca Merge pull request 'handmade dummy S3 local dev server' (#4) from nick12/hmn:hmns3 into localdev-2022
Reviewed-on: #4
2022-05-14 00:12:44 +00:00
Ben Visness 3aa16c6d12 Add owners to seeded projects 2022-05-11 23:39:43 -05:00
Ben Visness 196eda8185 Add README 2022-05-11 23:08:01 -05:00
Ben Visness a2ec57cf47 Seed projects 2022-05-11 22:24:05 -05:00
Ben Visness e4bb741a15 Automatically create the HMN user if necessary
This pretty much certainly won't work in real environments. If it does,
your db config is not secure :)
2022-05-11 19:50:51 -05:00
Ben Visness f4601198c9 Seed news posts 2022-05-07 14:45:21 -05:00
Ben Visness 3c4238994a Seed example forum threads 2022-05-07 14:31:37 -05:00
Ben Visness 3a93aa93e9 Seed users (and rework a lot of user access to use new helpers) 2022-05-07 13:58:00 -05:00
Ben Visness b46f5d8637 Add bare minimum seed 2022-05-07 11:37:15 -05:00
Ben Visness 1020039ea2 Fix up tests 2022-05-07 09:43:41 -05:00
Ben Visness 0e56f56372 Rename all the db tables 2022-05-07 08:11:05 -05:00
Ben Visness a147cfa325 Rework DB API
This takes advantage of generics, and generally clears up a lot of
inconsistencies and quality-of-life issues.

Start of db rework: clean up, start generics, improve tests

Write some nice aspirational package docs

Rework and document the db API

Tests still pass, at least...now for everything else

Update all callsites of db functions

Finish converting all callsites

Not too bad actually! Centralizing access into the helpers makes a big
difference.

wtf it works
2022-05-06 16:56:13 -05:00
unknown 69a44d1734 handmade dummy S3 local dev server 2022-05-01 02:41:58 -03:00
Ben Visness 6004149417 Fix zero-items bug with getPageInfo 2022-04-30 21:18:21 -05:00
66 changed files with 2888 additions and 1289 deletions

1
.gitignore vendored
View File

@ -13,3 +13,4 @@ hmn.conf
adminmailer/config.go
adminmailer/adminmailer
local/backups
/tmp

81
README.md Normal file
View File

@ -0,0 +1,81 @@
# Handmade Network
This is the codebase for the Handmade Network website, located online at https://handmade.network. The website is the home for all Handmade projects, the forums, the podcast, and other various initiatives.
The site is written in Go, and uses the standard library HTTP server with custom utilities for routing and middleware. It uses Postgres for its database, making heavy use of the excellent [pgx](https://github.com/jackc/pgx) library along with some custom wrappers. See the documentation in the `db` package for more details.
We want the website to be a great example of Handmade software on the web. We encourage you to read through the source, run it yourself, and learn from how we work. If you have questions about the code, or you're interested in contributing directly, reach out to us in #network-meta on the [Handmade Network Discord](https://discord.gg/hmn)!
## Prerequisites
You will need the following software installed:
- Go 1.18 or higher: https://go.dev/
You can download Go directly from the website, or install it through major package managers. If you already have Go installed, but are unsure of the version, you can check by running `go version`.
- Postgres: https://www.postgresql.org/
Any Postgres installation should work fine, although less common distributions may not work as nicely with our scripts out of the box. On Mac, [Postgres.app](https://postgresapp.com/) is recommended.
## First-time setup
- **Configure the site.** Copy `src/config/config.go.example` to `src/config/config.go`:
```
# On Windows
copy src\config\config.go.example src\config\config.go
# On Mac and Linux
cp src/config/config.go.example src/config/config.go
```
Depending on your installation of Postgres, you may need to modify the hostname and port in the Postgres section of the config.
- **Set up the database.** Run `go run src/main.go db seed` to initialize the database and fill it with sample data.
- **Update your hosts file.** The website uses subdomains for official projects, so the site cannot simply be run off `localhost`. Add the following
line to your hosts file:
```
127.0.0.1 handmade.local hero.handmade.local 4coder.handmade.local
```
You may need to edit the hosts file again in the future if you add more official projects.
On Windows, the hosts file is located at `C:\Windows\System32\Drivers\etc\hosts`. On Mac and Linux, the hosts file should be located at `/etc/hosts`. It can be edited using any plain text editor, but you will need superuser permissions.
## Running the site
Running the site is easy:
```
go run src/main.go
```
You should now be able to visit http://handmade.local:9001 to see the website!
There are also several other commands built into the website executable. You can see documentation for each of them by running `go run src/main.go help` or adding the `-h` flag to any command.
## Running tests
Also easy:
```
go test ./...
```
Note that we have a special coverage requirement for the `hmnurl` package. We use the tests in this package to ensure that our URL builder functions never go out of sync with the regexes used for routing. As a result, we mandate 100% coverage for all `Build` functions in `hmnurl`.
## Starter data
The `db seed` command pre-populates the site with realistic data. It also includes several user accounts that you may log into to test various situations:
| Username | Password |
| -------- | -------- |
| `admin` | `password` |
| `alice` | `password` |
| `bob` | `password` |
| `charlie` | `password` |
The `admin` user is a superuser on the site and will have access to all features, as well as the special admin UI for performing site maintenance and moderation. The other users are all normal users.

45
go.mod
View File

@ -1,10 +1,9 @@
module git.handmade.network/hmn/hmn
go 1.16
go 1.18
require (
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver v1.5.0 // indirect
github.com/HandmadeNetwork/golorem v0.0.0-20220507185207-414965a3a817
github.com/Masterminds/sprig v2.22.0+incompatible
github.com/alecthomas/chroma v0.9.2
github.com/aws/aws-sdk-go-v2 v1.8.1
@ -16,14 +15,11 @@ require (
github.com/go-stack/stack v1.8.0
github.com/google/uuid v1.2.0
github.com/gorilla/websocket v1.4.2
github.com/huandu/xstrings v1.3.2 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/jackc/pgconn v1.8.0
github.com/jackc/pgtype v1.6.2
github.com/jackc/pgx/v4 v4.10.1
github.com/jpillora/backoff v1.0.0
github.com/mitchellh/copystructure v1.1.1 // indirect
github.com/rs/zerolog v1.21.0
github.com/rs/zerolog v1.26.1
github.com/spf13/cobra v1.1.3
github.com/stretchr/testify v1.7.0
github.com/tcolgate/mp3 v0.0.0-20170426193717-e79c5a46d300
@ -31,10 +27,43 @@ require (
github.com/wellington/go-libsass v0.9.2
github.com/yuin/goldmark v1.4.1
github.com/yuin/goldmark-highlighting v0.0.0-20210516132338-9216f9c5aa01
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2
golang.org/x/crypto v0.0.0-20211215165025-cf75a172585e
golang.org/x/image v0.0.0-20210628002857-a66eb6448b8d
)
require (
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver v1.5.0 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.4.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.2.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.2.2 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.2.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.5.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.3.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.6.2 // indirect
github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dlclark/regexp2 v1.4.0 // indirect
github.com/huandu/xstrings v1.3.2 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/jackc/chunkreader/v2 v2.0.1 // indirect
github.com/jackc/pgio v1.0.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgproto3/v2 v2.0.6 // indirect
github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b // indirect
github.com/jackc/puddle v1.1.3 // indirect
github.com/mitchellh/copystructure v1.1.1 // indirect
github.com/mitchellh/reflectwalk v1.0.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d // indirect
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e // indirect
golang.org/x/text v0.3.6 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c // indirect
)
replace (
github.com/frustra/bbcode v0.0.0-20201127003707-6ef347fbe1c8 => github.com/HandmadeNetwork/bbcode v0.0.0-20210623031351-ec0e2e2e39d9
github.com/yuin/goldmark v1.4.1 => github.com/HandmadeNetwork/goldmark v1.4.1-0.20210707024600-f7e596e26b5e

33
go.sum
View File

@ -17,6 +17,8 @@ github.com/HandmadeNetwork/bbcode v0.0.0-20210623031351-ec0e2e2e39d9 h1:5WhEr56C
github.com/HandmadeNetwork/bbcode v0.0.0-20210623031351-ec0e2e2e39d9/go.mod h1:vMiNHD8absjmnO60Do5KCaJBwdbaiI/AzhMmSipMme4=
github.com/HandmadeNetwork/goldmark v1.4.1-0.20210707024600-f7e596e26b5e h1:z0GlF2OMmy852mrcMVpjZIzEHYCbUweS8RaWRCPfL1g=
github.com/HandmadeNetwork/goldmark v1.4.1-0.20210707024600-f7e596e26b5e/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/HandmadeNetwork/golorem v0.0.0-20220507185207-414965a3a817 h1:cBqVP/sLiK7DPay7Aac1PRUwu3fCVyL5Wc+xLXzqwkE=
github.com/HandmadeNetwork/golorem v0.0.0-20220507185207-414965a3a817/go.mod h1:doKbGBIdiM1nkEfvAeP5hvUmERah9H6StTVfCverqdE=
github.com/Masterminds/goutils v1.1.1 h1:5nUrii3FMTL5diU80unEVvNevw1nH4+ZV4DSLVJLSYI=
github.com/Masterminds/goutils v1.1.1/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU=
github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
@ -77,6 +79,7 @@ github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
@ -100,6 +103,7 @@ github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-stack/stack v1.8.0 h1:5SgMzNM5HxrEjV0ww2lTmX6E2Izsfxas4+YHWRs3Lsk=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/gofrs/uuid v3.2.0+incompatible h1:y12jRkkFxsd7GpqdSZ+/KCs/fJbqpEXSGd4+jfEaewE=
github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
@ -159,7 +163,6 @@ github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU=
github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA=
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/jackc/chunkreader v1.0.0 h1:4s39bBR8ByfqH+DKm8rQA3E1LHZWB9XWcrz8fqaZbe0=
github.com/jackc/chunkreader v1.0.0/go.mod h1:RT6O25fNZIuasFJRyZ4R/Y2BbhasbmZXF9QQ7T3kePo=
github.com/jackc/chunkreader/v2 v2.0.0/go.mod h1:odVSm741yZoC3dpHEUXIqA9tQRhFrgOHwnPIn9lDKlk=
github.com/jackc/chunkreader/v2 v2.0.1 h1:i+RDz65UE+mmpjTfyz0MoVTnzeYxroil2G82ki7MGG8=
@ -178,7 +181,6 @@ github.com/jackc/pgmock v0.0.0-20190831213851-13a1b77aafa2 h1:JVX6jT/XfzNqIjye47
github.com/jackc/pgmock v0.0.0-20190831213851-13a1b77aafa2/go.mod h1:fGZlG77KXmcq05nJLRkk0+p82V8B8Dw8KN2/V9c/OAE=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgproto3 v1.1.0 h1:FYYE4yRw+AgI8wXIinMlNjBbp/UitDJwfj5LqqewP1A=
github.com/jackc/pgproto3 v1.1.0/go.mod h1:eR5FA3leWg7p9aeAqi37XOTgTIbkABlvcPB3E5rlc78=
github.com/jackc/pgproto3/v2 v2.0.0-alpha1.0.20190420180111-c116219b62db/go.mod h1:bhq50y+xrl9n5mRYyCBFKkpRVTLYJVWeCc+mEAI3yXA=
github.com/jackc/pgproto3/v2 v2.0.0-alpha1.0.20190609003834-432c2951c711/go.mod h1:uH0AWtUmuShn0bcesswc4aBTWGvw0cAxIJp+6OB//Wg=
@ -289,10 +291,11 @@ github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40T
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ=
github.com/rs/xid v1.3.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
github.com/rs/zerolog v1.13.0/go.mod h1:YbFCdg8HfsridGWAh22vktObvhZbQsZXe4/zB0OKkWU=
github.com/rs/zerolog v1.15.0/go.mod h1:xYTKnLHcpfU2225ny5qZjxnj9NvkumZYjJHlAThCjNc=
github.com/rs/zerolog v1.21.0 h1:Q3vdXlfLNT+OftyBHsU0Y445MD+8m8axjKgf2si0QcM=
github.com/rs/zerolog v1.21.0/go.mod h1:ZPhntP/xmq1nnND05hhpAh2QMhSsA4UN3MGZ6O2J3hM=
github.com/rs/zerolog v1.26.1 h1:/ihwxqH+4z8UxyI70wM1z9yCvkWcfz/a3mj48k/Zngc=
github.com/rs/zerolog v1.26.1/go.mod h1:/wSSJWX7lVrsOwlbyTRSOJvqRlc+WjWlfes+CiJ+tmc=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
@ -338,8 +341,8 @@ github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1
github.com/wellington/go-libsass v0.9.2 h1:6Ims04UDdBs6/CGSVK5JC8FNikR5ssrsMMKE/uaO5Q8=
github.com/wellington/go-libsass v0.9.2/go.mod h1:mxgxgam0N0E+NAUMHLcu20Ccfc3mVpDkyrLDayqfiTs=
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.3.6/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/goldmark-highlighting v0.0.0-20210516132338-9216f9c5aa01 h1:0SJnXjE4jDClMW6grE0xpNhwpqbPwkBTn8zpVw5C0SI=
github.com/yuin/goldmark-highlighting v0.0.0-20210516132338-9216f9c5aa01/go.mod h1:TwKQPa5XkCCRC2GRZ5wtfNUTQ2+9/i19mGRijFeJ4BE=
github.com/zenazn/goji v0.9.0/go.mod h1:7S9M489iMyHBNxwZnk9/EHS098H4/F6TATF2mIxtB1Q=
@ -365,8 +368,8 @@ golang.org/x/crypto v0.0.0-20190911031432-227b76d455e7/go.mod h1:yigFU9vqHzYiE8U
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200323165209-0ec3e9974c59/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 h1:It14KIkyBFYkHkwZ7k45minvA9aorojkyjGk9KJ5B/w=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20211215165025-cf75a172585e h1:1SzTfNOXwIS2oWiMF+6qu0OUDKb0dauo6MoDUQyu+yU=
golang.org/x/crypto v0.0.0-20211215165025-cf75a172585e/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@ -387,7 +390,7 @@ golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@ -403,9 +406,9 @@ golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110 h1:qWPm9rbaAMKs8Bq/9LRpbMqxWRVUAQwMI9fVrssnTfw=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d h1:20cMwl2fHAzkJMEA+8J4JgqBQcQGzbisXo31MIeenXI=
golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@ -414,7 +417,7 @@ golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@ -437,10 +440,10 @@ golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200413165638-669c56c373c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4 h1:myAQVi0cGEoqQVR5POX+8RR2mrocKqNN1hmeMqhX27k=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e h1:WUoyKPm6nCo1BnNUvPGnFG3T5DUVem42yDJZZ4CNxMA=
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@ -472,7 +475,7 @@ golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtn
golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/tools v0.1.7/go.mod h1:LGqMHiF4EqQNHR1JncWGqT5BVaXmza+X+BDGol+dOxo=
golang.org/x/xerrors v0.0.0-20190410155217-1f06c39b4373/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20190513163551-3ee3066db522/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

View File

@ -1,23 +0,0 @@
#!/bin/bash
set -euxo pipefail
# This script is for use in local development only. It wipes the existing db,
# creates a new empty one, runs the initial migration to create the schema,
# and then imports actual db content on top of that.
# TODO(opensource): We should adapt Asaf's seedfile command and then delete this.
THIS_PATH=$(pwd)
BETA_PATH='/mnt/c/Users/bvisn/Developer/handmade/handmade-beta'
# BETA_PATH='/Users/benvisness/Developer/handmade/handmade-beta'
pushd $BETA_PATH
docker-compose down -v
docker-compose up -d postgres s3
sleep 3
docker-compose exec postgres bash -c "psql -U postgres -c \"CREATE ROLE hmn CREATEDB LOGIN PASSWORD 'password';\""
popd
go run src/main.go seedfile local/backups/hmn_pg_dump_live_2021-11-14
# go run src/main.go seedfile local/backups/hmn_pg_dump_live_2021-10-23

View File

@ -30,7 +30,7 @@ echo "Running migrations..."
systemctl stop hmn
do_as hmn <<'SCRIPT'
set -euo pipefail
/home/hmn/bin/hmn migrate
/home/hmn/bin/hmn db migrate
SCRIPT
systemctl start hmn

View File

@ -377,8 +377,8 @@ ${BLUE_BOLD}Download and restore a database backup${RESET}
su hmn
cd ~
hmn seedfile <your backup file>
hmn migrate
hmn db seedfile <your backup file>
hmn db migrate
${BLUE_BOLD}Restore static files${RESET}

View File

@ -37,8 +37,8 @@ func addCreateProjectCommand(projectCommand *cobra.Command) {
descParsed := parsing.ParseMarkdown(description, parsing.ForumRealMarkdown)
ctx := context.Background()
conn := db.NewConnPool(1, 1)
defer conn.Close()
conn := db.NewConn()
defer conn.Close(ctx)
tx, err := conn.Begin(ctx)
if err != nil {
@ -55,9 +55,9 @@ func addCreateProjectCommand(projectCommand *cobra.Command) {
}
hmn := p.Project
newProjectID, err := db.QueryInt(ctx, tx,
newProjectID, err := db.QueryOneScalar[int](ctx, tx,
`
INSERT INTO handmade_project (
INSERT INTO project (
slug,
name,
blurb,
@ -121,7 +121,7 @@ func addCreateProjectCommand(projectCommand *cobra.Command) {
for _, userID := range userIDs {
_, err := tx.Exec(ctx,
`
INSERT INTO handmade_user_projects (user_id, project_id)
INSERT INTO user_project (user_id, project_id)
VALUES ($1, $2)
`,
userID,
@ -160,8 +160,8 @@ func addProjectTagCommand(projectCommand *cobra.Command) {
tag, _ := cmd.Flags().GetString("tag")
ctx := context.Background()
conn := db.NewConnPool(1, 1)
defer conn.Close()
conn := db.NewConn()
defer conn.Close(ctx)
resultTag, err := hmndata.SetProjectTag(ctx, conn, nil, projectID, tag)
if err != nil {

View File

@ -42,10 +42,10 @@ func init() {
password := args[1]
ctx := context.Background()
conn := db.NewConnPool(1, 1)
defer conn.Close()
conn := db.NewConn()
defer conn.Close(ctx)
row := conn.QueryRow(ctx, "SELECT id, username FROM auth_user WHERE lower(username) = lower($1)", username)
row := conn.QueryRow(ctx, "SELECT id, username FROM hmn_user WHERE lower(username) = lower($1)", username)
var id int
var canonicalUsername string
err := row.Scan(&id, &canonicalUsername)
@ -83,10 +83,10 @@ func init() {
username := args[0]
ctx := context.Background()
conn := db.NewConnPool(1, 1)
defer conn.Close()
conn := db.NewConn()
defer conn.Close(ctx)
res, err := conn.Exec(ctx, "UPDATE auth_user SET status = $1 WHERE LOWER(username) = LOWER($2);", models.UserStatusConfirmed, username)
res, err := conn.Exec(ctx, "UPDATE hmn_user SET status = $1 WHERE LOWER(username) = LOWER($2);", models.UserStatusConfirmed, username)
if err != nil {
panic(err)
}
@ -138,10 +138,10 @@ func init() {
}
ctx := context.Background()
conn := db.NewConnPool(1, 1)
defer conn.Close()
conn := db.NewConn()
defer conn.Close(ctx)
res, err := conn.Exec(ctx, "UPDATE auth_user SET status = $1 WHERE LOWER(username) = LOWER($2);", status, username)
res, err := conn.Exec(ctx, "UPDATE hmn_user SET status = $1 WHERE LOWER(username) = LOWER($2);", status, username)
if err != nil {
panic(err)
}
@ -201,8 +201,8 @@ func init() {
projectSlug, _ := cmd.Flags().GetString("project_slug")
ctx := context.Background()
conn := db.NewConnPool(1, 1)
defer conn.Close()
conn := db.NewConn()
defer conn.Close(ctx)
tx, err := conn.Begin(ctx)
if err != nil {
@ -210,7 +210,7 @@ func init() {
}
defer tx.Rollback(ctx)
projectId, err := db.QueryInt(ctx, tx, `SELECT id FROM handmade_project WHERE slug = $1`, projectSlug)
projectId, err := db.QueryOneScalar[int](ctx, tx, `SELECT id FROM project WHERE slug = $1`, projectSlug)
if err != nil {
panic(err)
}
@ -218,8 +218,8 @@ func init() {
var parentId *int
if parentSlug == "" {
// Select the root subforum
id, err := db.QueryInt(ctx, tx,
`SELECT id FROM handmade_subforum WHERE parent_id IS NULL AND project_id = $1`,
id, err := db.QueryOneScalar[int](ctx, tx,
`SELECT id FROM subforum WHERE parent_id IS NULL AND project_id = $1`,
projectId,
)
if err != nil {
@ -228,8 +228,8 @@ func init() {
parentId = &id
} else {
// Select the parent
id, err := db.QueryInt(ctx, tx,
`SELECT id FROM handmade_subforum WHERE slug = $1 AND project_id = $2`,
id, err := db.QueryOneScalar[int](ctx, tx,
`SELECT id FROM subforum WHERE slug = $1 AND project_id = $2`,
parentSlug, projectId,
)
if err != nil {
@ -238,9 +238,9 @@ func init() {
parentId = &id
}
newId, err := db.QueryInt(ctx, tx,
newId, err := db.QueryOneScalar[int](ctx, tx,
`
INSERT INTO handmade_subforum (name, slug, blurb, parent_id, project_id)
INSERT INTO subforum (name, slug, blurb, parent_id, project_id)
VALUES ($1, $2, $3, $4, $5)
RETURNING id
`,
@ -280,8 +280,8 @@ func init() {
subforumSlug, _ := cmd.Flags().GetString("subforum_slug")
ctx := context.Background()
conn := db.NewConnPool(1, 1)
defer conn.Close()
conn := db.NewConn()
defer conn.Close(ctx)
tx, err := conn.Begin(ctx)
if err != nil {
@ -289,13 +289,13 @@ func init() {
}
defer tx.Rollback(ctx)
projectId, err := db.QueryInt(ctx, tx, `SELECT id FROM handmade_project WHERE slug = $1`, projectSlug)
projectId, err := db.QueryOneScalar[int](ctx, tx, `SELECT id FROM project WHERE slug = $1`, projectSlug)
if err != nil {
panic(err)
}
subforumId, err := db.QueryInt(ctx, tx,
`SELECT id FROM handmade_subforum WHERE slug = $1 AND project_id = $2`,
subforumId, err := db.QueryOneScalar[int](ctx, tx,
`SELECT id FROM subforum WHERE slug = $1 AND project_id = $2`,
subforumSlug, projectId,
)
if err != nil {
@ -314,7 +314,7 @@ func init() {
threadsTag, err := tx.Exec(ctx,
`
UPDATE handmade_thread
UPDATE thread
SET
project_id = $2,
subforum_id = $3,
@ -330,7 +330,7 @@ func init() {
postsTag, err := tx.Exec(ctx,
`
UPDATE handmade_post
UPDATE post
SET
thread_type = 2
WHERE

View File

@ -67,7 +67,7 @@ func SanitizeFilename(filename string) string {
}
func AssetKey(id, filename string) string {
return fmt.Sprintf("%s%s/%s", config.Config.DigitalOcean.AssetsPathPrefix, id, filename)
return fmt.Sprintf("%s/%s", id, filename)
}
type InvalidAssetError error
@ -122,7 +122,7 @@ func Create(ctx context.Context, dbConn db.ConnOrTx, in CreateInput) (*models.As
// TODO(db): Would be convient to use RETURNING here...
_, err = dbConn.Exec(ctx,
`
INSERT INTO handmade_asset (id, s3_key, filename, size, mime_type, sha1sum, width, height, uploader_id)
INSERT INTO asset (id, s3_key, filename, size, mime_type, sha1sum, width, height, uploader_id)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
`,
id,
@ -140,10 +140,10 @@ func Create(ctx context.Context, dbConn db.ConnOrTx, in CreateInput) (*models.As
}
// Fetch and return the new record
iasset, err := db.QueryOne(ctx, dbConn, models.Asset{},
asset, err := db.QueryOne[models.Asset](ctx, dbConn,
`
SELECT $columns
FROM handmade_asset
FROM asset
WHERE id = $1
`,
id,
@ -152,5 +152,5 @@ func Create(ctx context.Context, dbConn db.ConnOrTx, in CreateInput) (*models.As
return nil, oops.New(err, "failed to fetch newly-created asset")
}
return iasset.(*models.Asset), nil
return asset, nil
}

View File

@ -14,6 +14,7 @@ import (
"time"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/jobs"
"git.handmade.network/hmn/hmn/src/logging"
"git.handmade.network/hmn/hmn/src/models"
"git.handmade.network/hmn/hmn/src/oops"
@ -180,7 +181,7 @@ func HashPassword(password string) HashedPassword {
var ErrUserDoesNotExist = errors.New("user does not exist")
func UpdatePassword(ctx context.Context, conn db.ConnOrTx, username string, hp HashedPassword) error {
tag, err := conn.Exec(ctx, "UPDATE auth_user SET password = $1 WHERE username = $2", hp.String(), username)
tag, err := conn.Exec(ctx, "UPDATE hmn_user SET password = $1 WHERE username = $2", hp.String(), username)
if err != nil {
return oops.New(err, "failed to update password")
} else if tag.RowsAffected() < 1 {
@ -190,13 +191,18 @@ func UpdatePassword(ctx context.Context, conn db.ConnOrTx, username string, hp H
return nil
}
func SetPassword(ctx context.Context, conn db.ConnOrTx, username string, password string) error {
hp := HashPassword(password)
return UpdatePassword(ctx, conn, username, hp)
}
func DeleteInactiveUsers(ctx context.Context, conn *pgxpool.Pool) (int64, error) {
tag, err := conn.Exec(ctx,
`
DELETE FROM auth_user
DELETE FROM hmn_user
WHERE
status = $1 AND
(SELECT COUNT(*) as ct FROM handmade_onetimetoken AS ott WHERE ott.owner_id = auth_user.id AND ott.expires < $2 AND ott.token_type = $3) > 0;
(SELECT COUNT(*) as ct FROM one_time_token AS ott WHERE ott.owner_id = hmn_user.id AND ott.expires < $2 AND ott.token_type = $3) > 0;
`,
models.UserStatusInactive,
time.Now(),
@ -213,7 +219,7 @@ func DeleteInactiveUsers(ctx context.Context, conn *pgxpool.Pool) (int64, error)
func DeleteExpiredPasswordResets(ctx context.Context, conn *pgxpool.Pool) (int64, error) {
tag, err := conn.Exec(ctx,
`
DELETE FROM handmade_onetimetoken
DELETE FROM one_time_token
WHERE
token_type = $1
AND expires < $2
@ -229,10 +235,10 @@ func DeleteExpiredPasswordResets(ctx context.Context, conn *pgxpool.Pool) (int64
return tag.RowsAffected(), nil
}
func PeriodicallyDeleteInactiveUsers(ctx context.Context, conn *pgxpool.Pool) <-chan struct{} {
done := make(chan struct{})
func PeriodicallyDeleteInactiveUsers(ctx context.Context, conn *pgxpool.Pool) jobs.Job {
job := jobs.New()
go func() {
defer close(done)
defer job.Done()
t := time.NewTicker(1 * time.Hour)
for {
@ -260,5 +266,5 @@ func PeriodicallyDeleteInactiveUsers(ctx context.Context, conn *pgxpool.Pool) <-
}
}
}()
return done
return job
}

View File

@ -11,6 +11,7 @@ import (
"git.handmade.network/hmn/hmn/src/config"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/jobs"
"git.handmade.network/hmn/hmn/src/logging"
"git.handmade.network/hmn/hmn/src/models"
"git.handmade.network/hmn/hmn/src/oops"
@ -45,7 +46,7 @@ func makeCSRFToken() string {
var ErrNoSession = errors.New("no session found")
func GetSession(ctx context.Context, conn *pgxpool.Pool, id string) (*models.Session, error) {
row, err := db.QueryOne(ctx, conn, models.Session{}, "SELECT $columns FROM sessions WHERE id = $1", id)
sess, err := db.QueryOne[models.Session](ctx, conn, "SELECT $columns FROM session WHERE id = $1", id)
if err != nil {
if errors.Is(err, db.NotFound) {
return nil, ErrNoSession
@ -53,7 +54,6 @@ func GetSession(ctx context.Context, conn *pgxpool.Pool, id string) (*models.Ses
return nil, oops.New(err, "failed to get session")
}
}
sess := row.(*models.Session)
return sess, nil
}
@ -67,7 +67,7 @@ func CreateSession(ctx context.Context, conn *pgxpool.Pool, username string) (*m
}
_, err := conn.Exec(ctx,
"INSERT INTO sessions (id, username, expires_at, csrf_token) VALUES ($1, $2, $3, $4)",
"INSERT INTO session (id, username, expires_at, csrf_token) VALUES ($1, $2, $3, $4)",
session.ID, session.Username, session.ExpiresAt, session.CSRFToken,
)
if err != nil {
@ -80,7 +80,7 @@ func CreateSession(ctx context.Context, conn *pgxpool.Pool, username string) (*m
// Deletes a session by id. If no session with that id exists, no
// error is returned.
func DeleteSession(ctx context.Context, conn *pgxpool.Pool, id string) error {
_, err := conn.Exec(ctx, "DELETE FROM sessions WHERE id = $1", id)
_, err := conn.Exec(ctx, "DELETE FROM session WHERE id = $1", id)
if err != nil {
return oops.New(err, "failed to delete session")
}
@ -91,7 +91,7 @@ func DeleteSession(ctx context.Context, conn *pgxpool.Pool, id string) error {
func DeleteSessionForUser(ctx context.Context, conn *pgxpool.Pool, username string) error {
_, err := conn.Exec(ctx,
`
DELETE FROM sessions
DELETE FROM session
WHERE LOWER(username) = LOWER($1)
`,
username,
@ -125,7 +125,7 @@ var DeleteSessionCookie = &http.Cookie{
}
func DeleteExpiredSessions(ctx context.Context, conn *pgxpool.Pool) (int64, error) {
tag, err := conn.Exec(ctx, "DELETE FROM sessions WHERE expires_at <= CURRENT_TIMESTAMP")
tag, err := conn.Exec(ctx, "DELETE FROM session WHERE expires_at <= CURRENT_TIMESTAMP")
if err != nil {
return 0, oops.New(err, "failed to delete expired sessions")
}
@ -133,10 +133,10 @@ func DeleteExpiredSessions(ctx context.Context, conn *pgxpool.Pool) (int64, erro
return tag.RowsAffected(), nil
}
func PeriodicallyDeleteExpiredSessions(ctx context.Context, conn *pgxpool.Pool) <-chan struct{} {
done := make(chan struct{})
func PeriodicallyDeleteExpiredSessions(ctx context.Context, conn *pgxpool.Pool) jobs.Job {
job := jobs.New()
go func() {
defer close(done)
defer job.Done()
t := time.NewTicker(1 * time.Minute)
for {
@ -155,5 +155,5 @@ func PeriodicallyDeleteExpiredSessions(ctx context.Context, conn *pgxpool.Pool)
}
}
}()
return done
return job
}

View File

@ -7,14 +7,14 @@ import (
var Config = HMNConfig{
Env: Dev,
Addr: ":9001",
PrivateAddr: ":9002",
Addr: "localhost:9001",
PrivateAddr: "localhost:9002",
BaseUrl: "http://handmade.local:9001",
LogLevel: zerolog.TraceLevel, // InfoLevel is recommended for production
Postgres: PostgresConfig{
User: "hmn",
Password: "password",
Hostname: "handmade.local",
Hostname: "localhost",
Port: 5432,
DbName: "hmn",
LogLevel: pgx.LogLevelTrace, // LogLevelWarn is recommended for production
@ -30,21 +30,29 @@ var Config = HMNConfig{
AtomPassword: "password",
},
Email: EmailConfig{
ServerAddress: "smtp.example.com",
ServerPort: 587,
FromAddress: "noreply@example.com",
FromAddressPassword: "",
FromName: "Handmade Network Team",
OverrideRecipientEmail: "override@handmade.network", // NOTE(asaf): If this is not empty, all emails will be redirected to this address.
ServerAddress: "smtp.example.com",
ServerPort: 587,
FromAddress: "noreply@example.com",
FromAddressPassword: "",
FromName: "Handmade Network Team",
ForceToAddress: "localdev@example.com", // NOTE(asaf): If this is not empty, all emails will be sent to this address.
},
DigitalOcean: DigitalOceanConfig{
AssetsSpacesKey: "",
AssetsSpacesSecret: "",
AssetsSpacesRegion: "",
AssetsSpacesEndpoint: "",
AssetsSpacesBucket: "",
AssetsPathPrefix: "", // Empty is fine for production, but may be necessary for dev
AssetsPublicUrlRoot: "", // e.g. "https://bucket-name.region.cdn.digitaloceanspaces.com/". Note the trailing slash...
AssetsSpacesKey: "dummy",
AssetsSpacesSecret: "dummy",
AssetsSpacesRegion: "dummy",
AssetsSpacesEndpoint: "http://handmade.local:9003/",
AssetsSpacesBucket: "assets",
AssetsPublicUrlRoot: "http://handmade.local:9003/assets/",
// In prod, AssetsPublicUrlRoot will probably look something like:
//
// "https://bucket-name.region.cdn.digitaloceanspaces.com/"
//
// Note the trailing slash...
RunFakeServer: true,
FakeAddr: "localhost:9003",
},
Discord: DiscordConfig{
BotToken: "",

View File

@ -53,17 +53,19 @@ type DigitalOceanConfig struct {
AssetsSpacesRegion string
AssetsSpacesEndpoint string
AssetsSpacesBucket string
AssetsPathPrefix string
AssetsPublicUrlRoot string
RunFakeServer bool
FakeAddr string
}
type EmailConfig struct {
ServerAddress string
ServerPort int
FromAddress string
FromAddressPassword string
FromName string
OverrideRecipientEmail string
ServerAddress string
ServerPort int
FromAddress string
FromAddressPassword string
FromName string
ForceToAddress string
}
type DiscordConfig struct {

View File

@ -5,11 +5,13 @@ import (
"errors"
"fmt"
"reflect"
"regexp"
"strings"
"git.handmade.network/hmn/hmn/src/config"
"git.handmade.network/hmn/hmn/src/logging"
"git.handmade.network/hmn/hmn/src/oops"
"git.handmade.network/hmn/hmn/src/utils"
"github.com/google/uuid"
"github.com/jackc/pgconn"
"github.com/jackc/pgtype"
@ -20,46 +22,18 @@ import (
)
/*
Values of these kinds are ok to query even if they are not directly understood by pgtype.
This is common for custom types like:
type ThreadType int
A general error to be used when no results are found. This is the error returned
by QueryOne, and can generally be used by other database helpers that fetch a single
result but find nothing.
*/
var queryableKinds = []reflect.Kind{
reflect.Int,
}
/*
Checks if we are able to handle a particular type in a database query. This applies only to
primitive types and not structs, since the database only returns individual primitive types
and it is our job to stitch them back together into structs later.
*/
func typeIsQueryable(t reflect.Type) bool {
_, isRecognizedByPgtype := connInfo.DataTypeForValue(reflect.New(t).Elem().Interface()) // if pgtype recognizes it, we don't need to dig in further for more `db` tags
// NOTE: boy it would be nice if we didn't have to do reflect.New here, considering that pgtype is just doing reflection on the value anyway
if isRecognizedByPgtype {
return true
} else if t == reflect.TypeOf(uuid.UUID{}) {
return true
}
// pgtype doesn't recognize it, but maybe it's a primitive type we can deal with
k := t.Kind()
for _, qk := range queryableKinds {
if k == qk {
return true
}
}
return false
}
var NotFound = errors.New("not found")
// This interface should match both a direct pgx connection or a pgx transaction.
type ConnOrTx interface {
Query(ctx context.Context, sql string, args ...interface{}) (pgx.Rows, error)
QueryRow(ctx context.Context, sql string, args ...interface{}) pgx.Row
Exec(ctx context.Context, sql string, args ...interface{}) (pgconn.CommandTag, error)
Query(ctx context.Context, sql string, args ...any) (pgx.Rows, error)
QueryRow(ctx context.Context, sql string, args ...any) pgx.Row
Exec(ctx context.Context, sql string, args ...any) (pgconn.CommandTag, error)
CopyFrom(ctx context.Context, tableName pgx.Identifier, columnNames []string, rowSrc pgx.CopyFromSource) (int64, error)
// Both raw database connections and transactions in pgx can begin/commit
// transactions. For database connections it does the obvious thing; for
@ -70,8 +44,21 @@ type ConnOrTx interface {
var connInfo = pgtype.NewConnInfo()
// Creates a new connection to the HMN database.
// This connection is not safe for concurrent use.
func NewConn() *pgx.Conn {
conn, err := pgx.Connect(context.Background(), config.Config.Postgres.DSN())
return NewConnWithConfig(config.PostgresConfig{})
}
func NewConnWithConfig(cfg config.PostgresConfig) *pgx.Conn {
cfg = overrideDefaultConfig(cfg)
pgcfg, err := pgx.ParseConfig(cfg.DSN())
pgcfg.Logger = zerologadapter.NewLogger(log.Logger)
pgcfg.LogLevel = cfg.LogLevel
conn, err := pgx.ConnectConfig(context.Background(), pgcfg)
if err != nil {
panic(oops.New(err, "failed to connect to database"))
}
@ -79,15 +66,23 @@ func NewConn() *pgx.Conn {
return conn
}
func NewConnPool(minConns, maxConns int32) *pgxpool.Pool {
cfg, err := pgxpool.ParseConfig(config.Config.Postgres.DSN())
// Creates a connection pool for the HMN database.
// The resulting pool is safe for concurrent use.
func NewConnPool() *pgxpool.Pool {
return NewConnPoolWithConfig(config.PostgresConfig{})
}
cfg.MinConns = minConns
cfg.MaxConns = maxConns
cfg.ConnConfig.Logger = zerologadapter.NewLogger(log.Logger)
cfg.ConnConfig.LogLevel = config.Config.Postgres.LogLevel
func NewConnPoolWithConfig(cfg config.PostgresConfig) *pgxpool.Pool {
cfg = overrideDefaultConfig(cfg)
conn, err := pgxpool.ConnectConfig(context.Background(), cfg)
pgcfg, err := pgxpool.ParseConfig(cfg.DSN())
pgcfg.MinConns = cfg.MinConn
pgcfg.MaxConns = cfg.MaxConn
pgcfg.ConnConfig.Logger = zerologadapter.NewLogger(log.Logger)
pgcfg.ConnConfig.LogLevel = cfg.LogLevel
conn, err := pgxpool.ConnectConfig(context.Background(), pgcfg)
if err != nil {
panic(oops.New(err, "failed to create database connection pool"))
}
@ -95,144 +90,33 @@ func NewConnPool(minConns, maxConns int32) *pgxpool.Pool {
return conn
}
type StructQueryIterator struct {
fieldPaths [][]int
rows pgx.Rows
destType reflect.Type
closed chan struct{}
}
func (it *StructQueryIterator) Next() (interface{}, bool) {
hasNext := it.rows.Next()
if !hasNext {
it.Close()
return nil, false
}
result := reflect.New(it.destType)
vals, err := it.rows.Values()
if err != nil {
panic(err)
}
// Better logging of panics in this confusing reflection process
var currentField reflect.StructField
var currentValue reflect.Value
var currentIdx int
defer func() {
if r := recover(); r != nil {
if currentValue.IsValid() {
logging.Error().
Int("index", currentIdx).
Str("field name", currentField.Name).
Stringer("field type", currentField.Type).
Interface("value", currentValue.Interface()).
Stringer("value type", currentValue.Type()).
Msg("panic in iterator")
}
if currentField.Name != "" {
panic(fmt.Errorf("panic while processing field '%s': %v", currentField.Name, r))
} else {
panic(r)
}
}
}()
for i, val := range vals {
currentIdx = i
if val == nil {
continue
}
var field reflect.Value
field, currentField = followPathThroughStructs(result, it.fieldPaths[i])
if field.Kind() == reflect.Ptr {
field.Set(reflect.New(field.Type().Elem()))
field = field.Elem()
}
// Some actual values still come through as pointers (like net.IPNet). Dunno why.
// Regardless, we know it's not nil, so we can get at the contents.
valReflected := reflect.ValueOf(val)
if valReflected.Kind() == reflect.Ptr {
valReflected = valReflected.Elem()
}
currentValue = valReflected
switch field.Kind() {
case reflect.Int:
field.SetInt(valReflected.Int())
default:
field.Set(valReflected)
}
currentField = reflect.StructField{}
currentValue = reflect.Value{}
}
return result.Interface(), true
}
func (it *StructQueryIterator) Close() {
it.rows.Close()
select {
case it.closed <- struct{}{}:
default:
func overrideDefaultConfig(cfg config.PostgresConfig) config.PostgresConfig {
return config.PostgresConfig{
User: utils.OrDefault(cfg.User, config.Config.Postgres.User),
Password: utils.OrDefault(cfg.Password, config.Config.Postgres.Password),
Hostname: utils.OrDefault(cfg.Hostname, config.Config.Postgres.Hostname),
Port: utils.OrDefault(cfg.Port, config.Config.Postgres.Port),
DbName: utils.OrDefault(cfg.DbName, config.Config.Postgres.DbName),
LogLevel: utils.OrDefault(cfg.LogLevel, config.Config.Postgres.LogLevel),
MinConn: utils.OrDefault(cfg.MinConn, config.Config.Postgres.MinConn),
MaxConn: utils.OrDefault(cfg.MaxConn, config.Config.Postgres.MaxConn),
}
}
func (it *StructQueryIterator) ToSlice() []interface{} {
defer it.Close()
var result []interface{}
for {
row, ok := it.Next()
if !ok {
err := it.rows.Err()
if err != nil {
panic(oops.New(err, "error while iterating through db results"))
}
break
}
result = append(result, row)
}
return result
}
/*
Performs a SQL query and returns a slice of all the result rows. The query is just plain SQL, but make sure to read the package documentation for details. You must explicitly provide the type argument - this is how it knows what Go type to map the results to, and it cannot be inferred.
func followPathThroughStructs(structPtrVal reflect.Value, path []int) (reflect.Value, reflect.StructField) {
if len(path) < 1 {
panic(oops.New(nil, "can't follow an empty path"))
}
Any SQL query may be performed, including INSERT and UPDATE - as long as it returns a result set, you can use this. If the query does not return a result set, or you simply do not care about the result set, call Exec directly on your pgx connection.
if structPtrVal.Kind() != reflect.Ptr || structPtrVal.Elem().Kind() != reflect.Struct {
panic(oops.New(nil, "structPtrVal must be a pointer to a struct; got value of type %s", structPtrVal.Type()))
}
// more informative panic recovery
var field reflect.StructField
defer func() {
if r := recover(); r != nil {
panic(oops.New(nil, "panic at field '%s': %v", field.Name, r))
}
}()
val := structPtrVal
for _, i := range path {
if val.Kind() == reflect.Ptr && val.Type().Elem().Kind() == reflect.Struct {
if val.IsNil() {
val.Set(reflect.New(val.Type().Elem()))
}
val = val.Elem()
}
field = val.Type().Field(i)
val = val.Field(i)
}
return val, field
}
func Query(ctx context.Context, conn ConnOrTx, destExample interface{}, query string, args ...interface{}) ([]interface{}, error) {
it, err := QueryIterator(ctx, conn, destExample, query, args...)
This function always returns pointers to the values. This is convenient for structs, but for other types, you may wish to use QueryScalar.
*/
func Query[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) ([]*T, error) {
it, err := QueryIterator[T](ctx, conn, query, args...)
if err != nil {
return nil, err
} else {
@ -240,27 +124,163 @@ func Query(ctx context.Context, conn ConnOrTx, destExample interface{}, query st
}
}
func QueryIterator(ctx context.Context, conn ConnOrTx, destExample interface{}, query string, args ...interface{}) (*StructQueryIterator, error) {
destType := reflect.TypeOf(destExample)
columnNames, fieldPaths, err := getColumnNamesAndPaths(destType, nil, nil)
/*
Identical to Query, but panics if there was an error.
*/
func MustQuery[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) []*T {
result, err := Query[T](ctx, conn, query, args...)
if err != nil {
return nil, oops.New(err, "failed to generate column names")
panic(err)
}
return result
}
/*
Identical to Query, but returns only the first result row. If there are no
rows in the result set, returns NotFound.
*/
func QueryOne[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) (*T, error) {
rows, err := QueryIterator[T](ctx, conn, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
result, hasRow := rows.Next()
if !hasRow {
return nil, NotFound
}
columns := make([]string, 0, len(columnNames))
for _, strSlice := range columnNames {
tableName := strings.Join(strSlice[0:len(strSlice)-1], "_")
fullName := strSlice[len(strSlice)-1]
if tableName != "" {
fullName = tableName + "." + fullName
return result, nil
}
/*
Identical to QueryOne, but panics if there was an error.
*/
func MustQueryOne[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) *T {
result, err := QueryOne[T](ctx, conn, query, args...)
if err != nil {
panic(err)
}
return result
}
/*
Identical to Query, but returns concrete values instead of pointers. More convenient
for primitive types.
*/
func QueryScalar[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) ([]T, error) {
rows, err := QueryIterator[T](ctx, conn, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var result []T
for {
val, hasRow := rows.Next()
if !hasRow {
break
}
columns = append(columns, fullName)
result = append(result, *val)
}
columnNamesString := strings.Join(columns, ", ")
query = strings.Replace(query, "$columns", columnNamesString, -1)
return result, nil
}
rows, err := conn.Query(ctx, query, args...)
/*
Identical to QueryScalar, but panics if there was an error.
*/
func MustQueryScalar[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) []T {
result, err := QueryScalar[T](ctx, conn, query, args...)
if err != nil {
panic(err)
}
return result
}
/*
Identical to QueryScalar, but returns only the first result value. If there are
no rows in the result set, returns NotFound.
*/
func QueryOneScalar[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) (T, error) {
rows, err := QueryIterator[T](ctx, conn, query, args...)
if err != nil {
var zero T
return zero, err
}
defer rows.Close()
result, hasRow := rows.Next()
if !hasRow {
var zero T
return zero, NotFound
}
return *result, nil
}
/*
Identical to QueryOneScalar, but panics if there was an error.
*/
func MustQueryOneScalar[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) T {
result, err := QueryOneScalar[T](ctx, conn, query, args...)
if err != nil {
panic(err)
}
return result
}
/*
Identical to Query, but returns the ResultIterator instead of automatically converting the results to a slice. The iterator must be closed after use.
*/
func QueryIterator[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) (*Iterator[T], error) {
var destExample T
destType := reflect.TypeOf(destExample)
compiled := compileQuery(query, destType)
rows, err := conn.Query(ctx, compiled.query, args...)
if err != nil {
if errors.Is(err, context.DeadlineExceeded) {
panic("query exceeded its deadline")
@ -268,11 +288,12 @@ func QueryIterator(ctx context.Context, conn ConnOrTx, destExample interface{},
return nil, err
}
it := &StructQueryIterator{
fieldPaths: fieldPaths,
rows: rows,
destType: destType,
closed: make(chan struct{}, 1),
it := &Iterator[T]{
fieldPaths: compiled.fieldPaths,
rows: rows,
destType: compiled.destType,
destTypeIsScalar: typeIsQueryable(compiled.destType),
closed: make(chan struct{}, 1),
}
// Ensure that iterators are closed if context is cancelled. Otherwise, iterators can hold
@ -292,16 +313,88 @@ func QueryIterator(ctx context.Context, conn ConnOrTx, destExample interface{},
return it, nil
}
func getColumnNamesAndPaths(destType reflect.Type, pathSoFar []int, prefix []string) (names [][]string, paths [][]int, err error) {
var columnNames [][]string
var fieldPaths [][]int
/*
Identical to QueryIterator, but panics if there was an error.
*/
func MustQueryIterator[T any](
ctx context.Context,
conn ConnOrTx,
query string,
args ...any,
) *Iterator[T] {
result, err := QueryIterator[T](ctx, conn, query, args...)
if err != nil {
panic(err)
}
return result
}
// TODO: QueryFunc?
type compiledQuery struct {
query string
destType reflect.Type
fieldPaths []fieldPath
}
var reColumnsPlaceholder = regexp.MustCompile(`\$columns({(.*?)})?`)
func compileQuery(query string, destType reflect.Type) compiledQuery {
columnsMatch := reColumnsPlaceholder.FindStringSubmatch(query)
hasColumnsPlaceholder := columnsMatch != nil
if hasColumnsPlaceholder {
// The presence of the $columns placeholder means that the destination type
// must be a struct, and we will plonk that struct's fields into the query.
if destType.Kind() != reflect.Struct {
panic("$columns can only be used when querying into a struct")
}
var prefix []string
prefixText := columnsMatch[2]
if prefixText != "" {
prefix = []string{prefixText}
}
columnNames, fieldPaths := getColumnNamesAndPaths(destType, nil, prefix)
columns := make([]string, 0, len(columnNames))
for _, strSlice := range columnNames {
tableName := strings.Join(strSlice[0:len(strSlice)-1], "_")
fullName := strSlice[len(strSlice)-1]
if tableName != "" {
fullName = tableName + "." + fullName
}
columns = append(columns, fullName)
}
columnNamesString := strings.Join(columns, ", ")
query = reColumnsPlaceholder.ReplaceAllString(query, columnNamesString)
return compiledQuery{
query: query,
destType: destType,
fieldPaths: fieldPaths,
}
} else {
return compiledQuery{
query: query,
destType: destType,
}
}
}
func getColumnNamesAndPaths(destType reflect.Type, pathSoFar []int, prefix []string) (names []columnName, paths []fieldPath) {
var columnNames []columnName
var fieldPaths []fieldPath
if destType.Kind() == reflect.Ptr {
destType = destType.Elem()
}
if destType.Kind() != reflect.Struct {
return nil, nil, oops.New(nil, "can only get column names and paths from a struct, got type '%v' (at prefix '%v')", destType.Name(), prefix)
panic(fmt.Errorf("can only get column names and paths from a struct, got type '%v' (at prefix '%v')", destType.Name(), prefix))
}
type AnonPrefix struct {
@ -348,108 +441,214 @@ func getColumnNamesAndPaths(destType reflect.Type, pathSoFar []int, prefix []str
columnNames = append(columnNames, fieldColumnNames)
fieldPaths = append(fieldPaths, path)
} else if fieldType.Kind() == reflect.Struct {
subCols, subPaths, err := getColumnNamesAndPaths(fieldType, path, fieldColumnNames)
if err != nil {
return nil, nil, err
}
subCols, subPaths := getColumnNamesAndPaths(fieldType, path, fieldColumnNames)
columnNames = append(columnNames, subCols...)
fieldPaths = append(fieldPaths, subPaths...)
} else {
return nil, nil, oops.New(nil, "field '%s' in type %s has invalid type '%s'", field.Name, destType, field.Type)
panic(fmt.Errorf("field '%s' in type %s has invalid type '%s'", field.Name, destType, field.Type))
}
}
}
return columnNames, fieldPaths, nil
return columnNames, fieldPaths
}
/*
A general error to be used when no results are found. This is the error returned
by QueryOne, and can generally be used by other database helpers that fetch a single
result but find nothing.
Values of these kinds are ok to query even if they are not directly understood by pgtype.
This is common for custom types like:
type ThreadType int
*/
var NotFound = errors.New("not found")
func QueryOne(ctx context.Context, conn ConnOrTx, destExample interface{}, query string, args ...interface{}) (interface{}, error) {
rows, err := QueryIterator(ctx, conn, destExample, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
result, hasRow := rows.Next()
if !hasRow {
return nil, NotFound
}
return result, nil
var queryableKinds = []reflect.Kind{
reflect.Int,
}
func QueryScalar(ctx context.Context, conn ConnOrTx, query string, args ...interface{}) (interface{}, error) {
rows, err := conn.Query(ctx, query, args...)
if err != nil {
return nil, err
/*
Checks if we are able to handle a particular type in a database query. This applies only to
primitive types and not structs, since the database only returns individual primitive types
and it is our job to stitch them back together into structs later.
*/
func typeIsQueryable(t reflect.Type) bool {
_, isRecognizedByPgtype := connInfo.DataTypeForValue(reflect.New(t).Elem().Interface()) // if pgtype recognizes it, we don't need to dig in further for more `db` tags
// NOTE: boy it would be nice if we didn't have to do reflect.New here, considering that pgtype is just doing reflection on the value anyway
if isRecognizedByPgtype {
return true
} else if t == reflect.TypeOf(uuid.UUID{}) {
return true
}
defer rows.Close()
if rows.Next() {
vals, err := rows.Values()
if err != nil {
panic(err)
// pgtype doesn't recognize it, but maybe it's a primitive type we can deal with
k := t.Kind()
for _, qk := range queryableKinds {
if k == qk {
return true
}
}
return false
}
type columnName []string
// A path to a particular field in query's destination type. Each index in the slice
// corresponds to a field index for use with Field on a reflect.Type or reflect.Value.
type fieldPath []int
type Iterator[T any] struct {
fieldPaths []fieldPath
rows pgx.Rows
destType reflect.Type
destTypeIsScalar bool // NOTE(ben): Make sure this gets set every time destType gets set, based on typeIsQueryable(destType). This is kinda fragile...but also contained to this file, so doesn't seem worth a lazy evaluation or a constructor function.
closed chan struct{}
}
func (it *Iterator[T]) Next() (*T, bool) {
// TODO(ben): What happens if this panics? Does it leak resources? Do we need
// to put a recover() here and close the rows?
hasNext := it.rows.Next()
if !hasNext {
it.Close()
return nil, false
}
result := reflect.New(it.destType)
vals, err := it.rows.Values()
if err != nil {
panic(err)
}
if it.destTypeIsScalar {
// This type can be directly queried, meaning pgx recognizes it, it's
// a simple scalar thing, and we can just take the easy way out.
if len(vals) != 1 {
return nil, oops.New(nil, "you must query exactly one field with QueryScalar, not %v", len(vals))
panic(fmt.Errorf("tried to query a scalar value, but got %v values in the row", len(vals)))
}
setValueFromDB(result.Elem(), reflect.ValueOf(vals[0]))
return result.Interface().(*T), true
} else {
var currentField reflect.StructField
var currentValue reflect.Value
var currentIdx int
// Better logging of panics in this confusing reflection process
defer func() {
if r := recover(); r != nil {
if currentValue.IsValid() {
logging.Error().
Int("index", currentIdx).
Str("field name", currentField.Name).
Stringer("field type", currentField.Type).
Interface("value", currentValue.Interface()).
Stringer("value type", currentValue.Type()).
Msg("panic in iterator")
}
if currentField.Name != "" {
panic(fmt.Errorf("panic while processing field '%s': %v", currentField.Name, r))
} else {
panic(r)
}
}
}()
for i, val := range vals {
currentIdx = i
if val == nil {
continue
}
var field reflect.Value
field, currentField = followPathThroughStructs(result, it.fieldPaths[i])
if field.Kind() == reflect.Ptr {
field.Set(reflect.New(field.Type().Elem()))
field = field.Elem()
}
// Some actual values still come through as pointers (like net.IPNet). Dunno why.
// Regardless, we know it's not nil, so we can get at the contents.
valReflected := reflect.ValueOf(val)
if valReflected.Kind() == reflect.Ptr {
valReflected = valReflected.Elem()
}
currentValue = valReflected
setValueFromDB(field, valReflected)
currentField = reflect.StructField{}
currentValue = reflect.Value{}
}
return vals[0], nil
return result.Interface().(*T), true
}
return nil, NotFound
}
func QueryString(ctx context.Context, conn ConnOrTx, query string, args ...interface{}) (string, error) {
result, err := QueryScalar(ctx, conn, query, args...)
if err != nil {
return "", err
}
switch r := result.(type) {
case string:
return r, nil
func setValueFromDB(dest reflect.Value, value reflect.Value) {
switch dest.Kind() {
case reflect.Int:
dest.SetInt(value.Int())
default:
return "", oops.New(nil, "QueryString got a non-string result: %v", result)
dest.Set(value)
}
}
func QueryInt(ctx context.Context, conn ConnOrTx, query string, args ...interface{}) (int, error) {
result, err := QueryScalar(ctx, conn, query, args...)
if err != nil {
return 0, err
}
switch r := result.(type) {
case int:
return r, nil
case int32:
return int(r), nil
case int64:
return int(r), nil
func (it *Iterator[any]) Close() {
it.rows.Close()
select {
case it.closed <- struct{}{}:
default:
return 0, oops.New(nil, "QueryInt got a non-int result: %v", result)
}
}
func QueryBool(ctx context.Context, conn ConnOrTx, query string, args ...interface{}) (bool, error) {
result, err := QueryScalar(ctx, conn, query, args...)
if err != nil {
return false, err
/*
Pulls all the remaining values into a slice, and closes the iterator.
*/
func (it *Iterator[T]) ToSlice() []*T {
defer it.Close()
var result []*T
for {
row, ok := it.Next()
if !ok {
err := it.rows.Err()
if err != nil {
panic(oops.New(err, "error while iterating through db results"))
}
break
}
result = append(result, row)
}
return result
}
func followPathThroughStructs(structPtrVal reflect.Value, path []int) (reflect.Value, reflect.StructField) {
if len(path) < 1 {
panic(oops.New(nil, "can't follow an empty path"))
}
switch r := result.(type) {
case bool:
return r, nil
default:
return false, oops.New(nil, "QueryBool got a non-bool result: %v", result)
if structPtrVal.Kind() != reflect.Ptr || structPtrVal.Elem().Kind() != reflect.Struct {
panic(oops.New(nil, "structPtrVal must be a pointer to a struct; got value of type %s", structPtrVal.Type()))
}
// more informative panic recovery
var field reflect.StructField
defer func() {
if r := recover(); r != nil {
panic(oops.New(nil, "panic at field '%s': %v", field.Name, r))
}
}()
val := structPtrVal
for _, i := range path {
if val.Kind() == reflect.Ptr && val.Type().Elem().Kind() == reflect.Struct {
if val.IsNil() {
val.Set(reflect.New(val.Type().Elem()))
}
val = val.Elem()
}
field = val.Type().Field(i)
val = val.Field(i)
}
return val, field
}

View File

@ -10,52 +10,143 @@ import (
func TestPaths(t *testing.T) {
type CustomInt int
type S struct {
I int `db:"I"`
PI *int `db:"PI"`
CI CustomInt `db:"CI"`
PCI *CustomInt `db:"PCI"`
B bool `db:"B"`
PB *bool `db:"PB"`
type S2 struct {
B bool `db:"B"` // field 0
PB *bool `db:"PB"` // field 1
NoTag int
NoTag string // field 2
}
type S struct {
I int `db:"I"` // field 0
PI *int `db:"PI"` // field 1
CI CustomInt `db:"CI"` // field 2
PCI *CustomInt `db:"PCI"` // field 3
S2 `db:"S2"` // field 4 (embedded!)
PS2 *S2 `db:"PS2"` // field 5
NoTag int // field 6
}
type Nested struct {
S S `db:"S"`
PS *S `db:"PS"`
S S `db:"S"` // field 0
PS *S `db:"PS"` // field 1
NoTag S
NoTag S // field 2
}
type Embedded struct {
NoTag S
Nested
NoTag S // field 0
Nested // field 1
}
names, paths, err := getColumnNamesAndPaths(reflect.TypeOf(Embedded{}), nil, "")
if assert.Nil(t, err) {
assert.Equal(t, []string{
"S.I", "S.PI",
"S.CI", "S.PCI",
"S.B", "S.PB",
"PS.I", "PS.PI",
"PS.CI", "PS.PCI",
"PS.B", "PS.PB",
}, names)
assert.Equal(t, [][]int{
{1, 0, 0}, {1, 0, 1}, {1, 0, 2}, {1, 0, 3}, {1, 0, 4}, {1, 0, 5},
{1, 1, 0}, {1, 1, 1}, {1, 1, 2}, {1, 1, 3}, {1, 1, 4}, {1, 1, 5},
}, paths)
assert.True(t, len(names) == len(paths))
}
names, paths := getColumnNamesAndPaths(reflect.TypeOf(Embedded{}), nil, nil)
assert.Equal(t, []columnName{
{"S", "I"}, {"S", "PI"},
{"S", "CI"}, {"S", "PCI"},
{"S", "S2", "B"}, {"S", "S2", "PB"},
{"S", "PS2", "B"}, {"S", "PS2", "PB"},
{"PS", "I"}, {"PS", "PI"},
{"PS", "CI"}, {"PS", "PCI"},
{"PS", "S2", "B"}, {"PS", "S2", "PB"},
{"PS", "PS2", "B"}, {"PS", "PS2", "PB"},
}, names)
assert.Equal(t, []fieldPath{
{1, 0, 0}, {1, 0, 1}, // Nested.S.I, Nested.S.PI
{1, 0, 2}, {1, 0, 3}, // Nested.S.CI, Nested.S.PCI
{1, 0, 4, 0}, {1, 0, 4, 1}, // Nested.S.S2.B, Nested.S.S2.PB
{1, 0, 5, 0}, {1, 0, 5, 1}, // Nested.S.PS2.B, Nested.S.PS2.PB
{1, 1, 0}, {1, 1, 1}, // Nested.PS.I, Nested.PS.PI
{1, 1, 2}, {1, 1, 3}, // Nested.PS.CI, Nested.PS.PCI
{1, 1, 4, 0}, {1, 1, 4, 1}, // Nested.PS.S2.B, Nested.PS.S2.PB
{1, 1, 5, 0}, {1, 1, 5, 1}, // Nested.PS.PS2.B, Nested.PS.PS2.PB
}, paths)
assert.True(t, len(names) == len(paths))
testStruct := Embedded{}
for i, path := range paths {
val, field := followPathThroughStructs(reflect.ValueOf(&testStruct), path)
assert.True(t, val.IsValid())
assert.True(t, strings.Contains(names[i], field.Name))
assert.True(t, strings.Contains(names[i][len(names[i])-1], field.Name))
}
}
func TestCompileQuery(t *testing.T) {
t.Run("simple struct", func(t *testing.T) {
type Dest struct {
Foo int `db:"foo"`
Bar bool `db:"bar"`
Nope string // no tag
}
compiled := compileQuery("SELECT $columns FROM greeblies", reflect.TypeOf(Dest{}))
assert.Equal(t, "SELECT foo, bar FROM greeblies", compiled.query)
})
t.Run("complex structs", func(t *testing.T) {
type CustomInt int
type S2 struct {
B bool `db:"B"`
PB *bool `db:"PB"`
NoTag string
}
type S struct {
I int `db:"I"`
PI *int `db:"PI"`
CI CustomInt `db:"CI"`
PCI *CustomInt `db:"PCI"`
S2 `db:"S2"` // embedded!
PS2 *S2 `db:"PS2"`
NoTag int
}
type Nested struct {
S S `db:"S"`
PS *S `db:"PS"`
NoTag S
}
type Dest struct {
NoTag S
Nested
}
compiled := compileQuery("SELECT $columns FROM greeblies", reflect.TypeOf(Dest{}))
assert.Equal(t, "SELECT S.I, S.PI, S.CI, S.PCI, S_S2.B, S_S2.PB, S_PS2.B, S_PS2.PB, PS.I, PS.PI, PS.CI, PS.PCI, PS_S2.B, PS_S2.PB, PS_PS2.B, PS_PS2.PB FROM greeblies", compiled.query)
})
t.Run("int", func(t *testing.T) {
type Dest int
// There should be no error here because we do not need to extract columns from
// the destination type. There may be errors down the line in value iteration, but
// that is always the case if the Go types don't match the query.
compiled := compileQuery("SELECT id FROM greeblies", reflect.TypeOf(Dest(0)))
assert.Equal(t, "SELECT id FROM greeblies", compiled.query)
})
t.Run("just one table", func(t *testing.T) {
type Dest struct {
Foo int `db:"foo"`
Bar bool `db:"bar"`
Nope string // no tag
}
// The prefix is necessary because otherwise we would have to provide a struct with
// a db tag in order to provide the query with the `greeblies.` prefix in the
// final query. This comes up a lot when we do a JOIN to help with a condition, but
// don't actually care about any of the data we joined to.
compiled := compileQuery(
"SELECT $columns{greeblies} FROM greeblies NATURAL JOIN props",
reflect.TypeOf(Dest{}),
)
assert.Equal(t, "SELECT greeblies.foo, greeblies.bar FROM greeblies NATURAL JOIN props", compiled.query)
})
t.Run("using $columns without a struct is not allowed", func(t *testing.T) {
type Dest int
assert.Panics(t, func() {
compileQuery("SELECT $columns FROM greeblies", reflect.TypeOf(Dest(0)))
})
})
}
func TestQueryBuilder(t *testing.T) {
t.Run("happy time", func(t *testing.T) {
var qb QueryBuilder

59
src/db/doc.go Normal file
View File

@ -0,0 +1,59 @@
/*
This package contains lowish-level APIs for making database queries to our Postgres database. It streamlines the process of mapping query results to Go types, while allowing you to write arbitrary SQL queries.
The primary functions are Query and QueryIterator. See the package and function examples for detailed usage.
Query syntax
This package allows a few small extensions to SQL syntax to streamline the interaction between Go and Postgres.
Arguments can be provided using placeholders like $1, $2, etc. All arguments will be safely escaped and mapped from their Go type to the correct Postgres type. (This is a direct proxy to pgx.)
projectIDs, err := db.Query[int](ctx, conn,
`
SELECT id
FROM project
WHERE
slug = ANY($1)
AND hidden = $2
`,
[]string{"4coder", "metadesk"},
false,
)
(This also demonstrates a useful tip: if you want to use a slice in your query, use Postgres arrays instead of IN.)
When querying individual fields, you can simply select the field like so:
ids, err := db.Query[int](ctx, conn, `SELECT id FROM project`)
To query multiple columns at once, you may use a struct type with `db:"column_name"` tags, and the special $columns placeholder:
type Project struct {
ID int `db:"id"`
Slug string `db:"slug"`
DateCreated time.Time `db:"date_created"`
}
projects, err := db.Query[Project](ctx, conn, `SELECT $columns FROM ...`)
// Resulting query:
// SELECT id, slug, date_created FROM ...
Sometimes a table name prefix is required on each column to disambiguate between column names, especially when performing a JOIN. In those situations, you can include the prefix in the $columns placeholder like $columns{prefix}:
type Project struct {
ID int `db:"id"`
Slug string `db:"slug"`
DateCreated time.Time `db:"date_created"`
}
orphanedProjects, err := db.Query[Project](ctx, conn, `
SELECT $columns{projects}
FROM
project AS projects
LEFT JOIN user_project AS uproj
WHERE
uproj.user_id IS NULL
`)
// Resulting query:
// SELECT projects.id, projects.slug, projects.date_created FROM ...
*/
package db

View File

@ -7,7 +7,7 @@ import (
type QueryBuilder struct {
sql strings.Builder
args []interface{}
args []any
}
/*
@ -18,7 +18,7 @@ of `$?` will be replaced with the correct argument number.
foo ARG1 bar ARG2 baz $?
foo ARG1 bar ARG2 baz ARG3
*/
func (qb *QueryBuilder) Add(sql string, args ...interface{}) {
func (qb *QueryBuilder) Add(sql string, args ...any) {
numPlaceholders := strings.Count(sql, "$?")
if numPlaceholders != len(args) {
panic(fmt.Errorf("cannot add chunk to query; expected %d arguments but got %d", numPlaceholders, len(args)))
@ -37,6 +37,6 @@ func (qb *QueryBuilder) String() string {
return qb.sql.String()
}
func (qb *QueryBuilder) Args() []interface{} {
func (qb *QueryBuilder) Args() []any {
return qb.args
}

View File

@ -27,7 +27,7 @@ func init() {
Long: "Scrape the entire history of Discord channels, saving message content (but not creating snippets)",
Run: func(cmd *cobra.Command, args []string) {
ctx := context.Background()
conn := db.NewConnPool(1, 1)
conn := db.NewConnPool()
defer conn.Close()
for _, channelID := range args {
@ -47,8 +47,8 @@ func init() {
os.Exit(1)
}
ctx := context.Background()
conn := db.NewConnPool(1, 1)
defer conn.Close()
conn := db.NewConn()
defer conn.Close(ctx)
chanID := args[0]

View File

@ -90,20 +90,18 @@ func (bot *botInstance) handleProfileCommand(ctx context.Context, i *Interaction
return
}
type profileResult struct {
HMNUser models.User `db:"auth_user"`
}
ires, err := db.QueryOne(ctx, bot.dbConn, profileResult{},
hmnUser, err := db.QueryOne[models.User](ctx, bot.dbConn,
`
SELECT $columns
SELECT $columns{hmn_user}
FROM
handmade_discorduser AS duser
JOIN auth_user ON duser.hmn_user_id = auth_user.id
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
discord_user AS duser
JOIN hmn_user ON duser.hmn_user_id = hmn_user.id
WHERE
duser.userid = $1
AND hmn_user.status = $2
`,
userID,
models.UserStatusApproved,
)
if err != nil {
if errors.Is(err, db.NotFound) {
@ -122,16 +120,15 @@ func (bot *botInstance) handleProfileCommand(ctx context.Context, i *Interaction
}
return
}
res := ires.(*profileResult)
projectsAndStuff, err := hmndata.FetchProjects(ctx, bot.dbConn, nil, hmndata.ProjectsQuery{
OwnerIDs: []int{res.HMNUser.ID},
OwnerIDs: []int{hmnUser.ID},
})
if err != nil {
logging.ExtractLogger(ctx).Error().Err(err).Msg("failed to fetch user projects")
}
url := hmnurl.BuildUserProfile(res.HMNUser.Username)
url := hmnurl.BuildUserProfile(hmnUser.Username)
msg := fmt.Sprintf("<@%s>'s profile can be viewed at %s.", member.User.ID, url)
if len(projectsAndStuff) > 0 {
projectNoun := "projects"

View File

@ -13,6 +13,7 @@ import (
"git.handmade.network/hmn/hmn/src/config"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/jobs"
"git.handmade.network/hmn/hmn/src/logging"
"git.handmade.network/hmn/hmn/src/models"
"git.handmade.network/hmn/hmn/src/oops"
@ -22,22 +23,20 @@ import (
"github.com/jpillora/backoff"
)
func RunDiscordBot(ctx context.Context, dbConn *pgxpool.Pool) <-chan struct{} {
func RunDiscordBot(ctx context.Context, dbConn *pgxpool.Pool) jobs.Job {
log := logging.ExtractLogger(ctx).With().Str("module", "discord").Logger()
ctx = logging.AttachLoggerToContext(&log, ctx)
if config.Config.Discord.BotToken == "" {
log.Warn().Msg("No Discord bot token was provided, so the Discord bot cannot run.")
done := make(chan struct{}, 1)
done <- struct{}{}
return done
return jobs.Noop()
}
done := make(chan struct{})
job := jobs.New()
go func() {
defer func() {
log.Debug().Msg("shut down Discord bot")
done <- struct{}{}
job.Done()
}()
boff := backoff.Backoff{
@ -88,7 +87,7 @@ func RunDiscordBot(ctx context.Context, dbConn *pgxpool.Pool) <-chan struct{} {
}()
}
}()
return done
return job
}
var outgoingMessagesReady = make(chan struct{}, 1)
@ -250,7 +249,7 @@ func (bot *botInstance) connect(ctx context.Context) error {
// an old one or starting a new one.
shouldResume := true
isession, err := db.QueryOne(ctx, bot.dbConn, models.DiscordSession{}, `SELECT $columns FROM discord_session`)
session, err := db.QueryOne[models.DiscordSession](ctx, bot.dbConn, `SELECT $columns FROM discord_session`)
if err != nil {
if errors.Is(err, db.NotFound) {
// No session yet! Just identify and get on with it
@ -262,8 +261,6 @@ func (bot *botInstance) connect(ctx context.Context) error {
if shouldResume {
// Reconnect to the previous session
session := isession.(*models.DiscordSession)
err := bot.sendGatewayMessage(ctx, GatewayMessage{
Opcode: OpcodeResume,
Data: Resume{
@ -356,7 +353,7 @@ func (bot *botInstance) doSender(ctx context.Context) {
}
bot.didAckHeartbeat = false
latestSequenceNumber, err := db.QueryInt(ctx, bot.dbConn, `SELECT sequence_number FROM discord_session`)
latestSequenceNumber, err := db.QueryOneScalar[int](ctx, bot.dbConn, `SELECT sequence_number FROM discord_session`)
if err != nil {
log.Error().Err(err).Msg("failed to fetch latest sequence number from the db")
return false
@ -408,9 +405,9 @@ func (bot *botInstance) doSender(ctx context.Context) {
}
defer tx.Rollback(ctx)
msgs, err := db.Query(ctx, tx, models.DiscordOutgoingMessage{}, `
msgs, err := db.Query[models.DiscordOutgoingMessage](ctx, tx, `
SELECT $columns
FROM discord_outgoingmessages
FROM discord_outgoing_message
ORDER BY id ASC
`)
if err != nil {
@ -418,8 +415,7 @@ func (bot *botInstance) doSender(ctx context.Context) {
return
}
for _, imsg := range msgs {
msg := imsg.(*models.DiscordOutgoingMessage)
for _, msg := range msgs {
if time.Now().After(msg.ExpiresAt) {
continue
}
@ -433,7 +429,7 @@ func (bot *botInstance) doSender(ctx context.Context) {
https://www.postgresql.org/docs/current/transaction-iso.html
*/
_, err = tx.Exec(ctx, `DELETE FROM discord_outgoingmessages`)
_, err = tx.Exec(ctx, `DELETE FROM discord_outgoing_message`)
if err != nil {
log.Error().Err(err).Msg("failed to delete outgoing messages")
return
@ -647,7 +643,7 @@ func SendMessages(
_, err = tx.Exec(ctx,
`
INSERT INTO discord_outgoingmessages (channel_id, payload_json, expires_at)
INSERT INTO discord_outgoing_message (channel_id, payload_json, expires_at)
VALUES ($1, $2, $3)
`,
msg.ChannelID,

View File

@ -7,27 +7,26 @@ import (
"git.handmade.network/hmn/hmn/src/config"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/jobs"
"git.handmade.network/hmn/hmn/src/logging"
"git.handmade.network/hmn/hmn/src/models"
"github.com/jackc/pgx/v4/pgxpool"
)
func RunHistoryWatcher(ctx context.Context, dbConn *pgxpool.Pool) <-chan struct{} {
func RunHistoryWatcher(ctx context.Context, dbConn *pgxpool.Pool) jobs.Job {
log := logging.ExtractLogger(ctx).With().Str("discord goroutine", "history watcher").Logger()
ctx = logging.AttachLoggerToContext(&log, ctx)
if config.Config.Discord.BotToken == "" {
log.Warn().Msg("No Discord bot token was provided, so the Discord history bot cannot run.")
done := make(chan struct{}, 1)
done <- struct{}{}
return done
return jobs.Noop()
}
done := make(chan struct{})
job := jobs.New()
go func() {
defer func() {
log.Debug().Msg("shut down Discord history watcher")
done <- struct{}{}
job.Done()
}()
newUserTicker := time.NewTicker(5 * time.Second)
@ -67,22 +66,19 @@ func RunHistoryWatcher(ctx context.Context, dbConn *pgxpool.Pool) <-chan struct{
}
}()
return done
return job
}
func fetchMissingContent(ctx context.Context, dbConn *pgxpool.Pool) {
log := logging.ExtractLogger(ctx)
type query struct {
Message models.DiscordMessage `db:"msg"`
}
imessagesWithoutContent, err := db.Query(ctx, dbConn, query{},
messagesWithoutContent, err := db.Query[models.DiscordMessage](ctx, dbConn,
`
SELECT $columns
SELECT $columns{msg}
FROM
handmade_discordmessage AS msg
JOIN handmade_discorduser AS duser ON msg.user_id = duser.userid -- only fetch messages for linked discord users
LEFT JOIN handmade_discordmessagecontent AS c ON c.message_id = msg.id
discord_message AS msg
JOIN discord_user AS duser ON msg.user_id = duser.userid -- only fetch messages for linked discord users
LEFT JOIN discord_message_content AS c ON c.message_id = msg.id
WHERE
c.last_content IS NULL
AND msg.guild_id = $1
@ -95,10 +91,10 @@ func fetchMissingContent(ctx context.Context, dbConn *pgxpool.Pool) {
return
}
if len(imessagesWithoutContent) > 0 {
log.Info().Msgf("There are %d Discord messages without content, fetching their content now...", len(imessagesWithoutContent))
if len(messagesWithoutContent) > 0 {
log.Info().Msgf("There are %d Discord messages without content, fetching their content now...", len(messagesWithoutContent))
msgloop:
for _, imsg := range imessagesWithoutContent {
for _, msg := range messagesWithoutContent {
select {
case <-ctx.Done():
log.Info().Msg("Scrape was canceled")
@ -106,8 +102,6 @@ func fetchMissingContent(ctx context.Context, dbConn *pgxpool.Pool) {
default:
}
msg := imsg.(*query).Message
discordMsg, err := GetChannelMessage(ctx, msg.ChannelID, msg.ID)
if errors.Is(err, NotFound) {
// This message has apparently been deleted; delete it from our database

View File

@ -165,10 +165,10 @@ func InternMessage(
dbConn db.ConnOrTx,
msg *Message,
) error {
_, err := db.QueryOne(ctx, dbConn, models.DiscordMessage{},
_, err := db.QueryOne[models.DiscordMessage](ctx, dbConn,
`
SELECT $columns
FROM handmade_discordmessage
FROM discord_message
WHERE id = $1
`,
msg.ID,
@ -190,7 +190,7 @@ func InternMessage(
_, err = dbConn.Exec(ctx,
`
INSERT INTO handmade_discordmessage (id, channel_id, guild_id, url, user_id, sent_at, snippet_created)
INSERT INTO discord_message (id, channel_id, guild_id, url, user_id, sent_at, snippet_created)
VALUES ($1, $2, $3, $4, $5, $6, $7)
`,
msg.ID,
@ -219,15 +219,14 @@ type InternedMessage struct {
}
func FetchInternedMessage(ctx context.Context, dbConn db.ConnOrTx, msgId string) (*InternedMessage, error) {
result, err := db.QueryOne(ctx, dbConn, InternedMessage{},
interned, err := db.QueryOne[InternedMessage](ctx, dbConn,
`
SELECT $columns
FROM
handmade_discordmessage AS message
LEFT JOIN handmade_discordmessagecontent AS content ON content.message_id = message.id
LEFT JOIN handmade_discorduser AS duser ON duser.userid = message.user_id
LEFT JOIN auth_user AS hmnuser ON hmnuser.id = duser.hmn_user_id
LEFT JOIN handmade_asset AS hmnuser_avatar ON hmnuser_avatar.id = hmnuser.avatar_asset_id
discord_message AS message
LEFT JOIN discord_message_content AS content ON content.message_id = message.id
LEFT JOIN discord_user AS duser ON duser.userid = message.user_id
LEFT JOIN hmn_user AS hmnuser ON hmnuser.id = duser.hmn_user_id
WHERE message.id = $1
`,
msgId,
@ -235,8 +234,6 @@ func FetchInternedMessage(ctx context.Context, dbConn db.ConnOrTx, msgId string)
if err != nil {
return nil, err
}
interned := result.(*InternedMessage)
return interned, nil
}
@ -283,10 +280,10 @@ func HandleInternedMessage(ctx context.Context, dbConn db.ConnOrTx, msg *Message
}
func DeleteInternedMessage(ctx context.Context, dbConn db.ConnOrTx, interned *InternedMessage) error {
isnippet, err := db.QueryOne(ctx, dbConn, models.Snippet{},
snippet, err := db.QueryOne[models.Snippet](ctx, dbConn,
`
SELECT $columns
FROM handmade_snippet
FROM snippet
WHERE discord_message_id = $1
`,
interned.Message.ID,
@ -294,19 +291,15 @@ func DeleteInternedMessage(ctx context.Context, dbConn db.ConnOrTx, interned *In
if err != nil && !errors.Is(err, db.NotFound) {
return oops.New(err, "failed to fetch snippet for discord message")
}
var snippet *models.Snippet
if !errors.Is(err, db.NotFound) {
snippet = isnippet.(*models.Snippet)
}
// NOTE(asaf): Also deletes the following through a db cascade:
// * handmade_discordmessageattachment
// * handmade_discordmessagecontent
// * handmade_discordmessageembed
// * discord_message_attachment
// * discord_message_content
// * discord_message_embed
// DOES NOT DELETE ASSETS FOR CONTENT/EMBEDS
_, err = dbConn.Exec(ctx,
`
DELETE FROM handmade_discordmessage
DELETE FROM discord_message
WHERE id = $1
`,
interned.Message.ID,
@ -318,7 +311,7 @@ func DeleteInternedMessage(ctx context.Context, dbConn db.ConnOrTx, interned *In
// NOTE(asaf): Does not delete asset!
_, err = dbConn.Exec(ctx,
`
DELETE FROM handmade_snippet
DELETE FROM snippet
WHERE id = $1
`,
snippet.ID,
@ -353,7 +346,7 @@ func SaveMessageContents(
if msg.OriginalHasFields("content") {
_, err := dbConn.Exec(ctx,
`
INSERT INTO handmade_discordmessagecontent (message_id, discord_id, last_content)
INSERT INTO discord_message_content (message_id, discord_id, last_content)
VALUES ($1, $2, $3)
ON CONFLICT (message_id) DO UPDATE SET
discord_id = EXCLUDED.discord_id,
@ -367,20 +360,20 @@ func SaveMessageContents(
return oops.New(err, "failed to create or update message contents")
}
icontent, err := db.QueryOne(ctx, dbConn, models.DiscordMessageContent{},
content, err := db.QueryOne[models.DiscordMessageContent](ctx, dbConn,
`
SELECT $columns
FROM
handmade_discordmessagecontent
discord_message_content
WHERE
handmade_discordmessagecontent.message_id = $1
discord_message_content.message_id = $1
`,
interned.Message.ID,
)
if err != nil {
return oops.New(err, "failed to fetch message contents")
}
interned.MessageContent = icontent.(*models.DiscordMessageContent)
interned.MessageContent = content
}
// Save attachments
@ -395,12 +388,12 @@ func SaveMessageContents(
// Save / delete embeds
if msg.OriginalHasFields("embeds") {
numSavedEmbeds, err := db.QueryInt(ctx, dbConn,
numSavedEmbeds, err := db.QueryOneScalar[int](ctx, dbConn,
`
SELECT COUNT(*)
FROM handmade_discordmessageembed
WHERE message_id = $1
`,
SELECT COUNT(*)
FROM discord_message_embed
WHERE message_id = $1
`,
msg.ID,
)
if err != nil {
@ -418,7 +411,7 @@ func SaveMessageContents(
// Embeds were removed from the message
_, err := dbConn.Exec(ctx,
`
DELETE FROM handmade_discordmessageembed
DELETE FROM discord_message_embed
WHERE message_id = $1
`,
msg.ID,
@ -472,16 +465,16 @@ func saveAttachment(
hmnUserID int,
discordMessageID string,
) (*models.DiscordMessageAttachment, error) {
iexisting, err := db.QueryOne(ctx, tx, models.DiscordMessageAttachment{},
existing, err := db.QueryOne[models.DiscordMessageAttachment](ctx, tx,
`
SELECT $columns
FROM handmade_discordmessageattachment
FROM discord_message_attachment
WHERE id = $1
`,
attachment.ID,
)
if err == nil {
return iexisting.(*models.DiscordMessageAttachment), nil
return existing, nil
} else if errors.Is(err, db.NotFound) {
// this is fine, just create it
} else {
@ -523,7 +516,7 @@ func saveAttachment(
// TODO(db): RETURNING plz thanks
_, err = tx.Exec(ctx,
`
INSERT INTO handmade_discordmessageattachment (id, asset_id, message_id)
INSERT INTO discord_message_attachment (id, asset_id, message_id)
VALUES ($1, $2, $3)
`,
attachment.ID,
@ -534,10 +527,10 @@ func saveAttachment(
return nil, oops.New(err, "failed to save Discord attachment data")
}
iDiscordAttachment, err := db.QueryOne(ctx, tx, models.DiscordMessageAttachment{},
discordAttachment, err := db.QueryOne[models.DiscordMessageAttachment](ctx, tx,
`
SELECT $columns
FROM handmade_discordmessageattachment
FROM discord_message_attachment
WHERE id = $1
`,
attachment.ID,
@ -546,7 +539,7 @@ func saveAttachment(
return nil, oops.New(err, "failed to fetch new Discord attachment data")
}
return iDiscordAttachment.(*models.DiscordMessageAttachment), nil
return discordAttachment, nil
}
// Saves an embed from Discord. NOTE: This is _not_ idempotent, so only call it
@ -621,7 +614,7 @@ func saveEmbed(
var savedEmbedId int
err = tx.QueryRow(ctx,
`
INSERT INTO handmade_discordmessageembed (title, description, url, message_id, image_id, video_id)
INSERT INTO discord_message_embed (title, description, url, message_id, image_id, video_id)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id
`,
@ -636,10 +629,10 @@ func saveEmbed(
return nil, oops.New(err, "failed to insert new embed")
}
iDiscordEmbed, err := db.QueryOne(ctx, tx, models.DiscordMessageEmbed{},
discordEmbed, err := db.QueryOne[models.DiscordMessageEmbed](ctx, tx,
`
SELECT $columns
FROM handmade_discordmessageembed
FROM discord_message_embed
WHERE id = $1
`,
savedEmbedId,
@ -648,14 +641,14 @@ func saveEmbed(
return nil, oops.New(err, "failed to fetch new Discord embed data")
}
return iDiscordEmbed.(*models.DiscordMessageEmbed), nil
return discordEmbed, nil
}
func FetchSnippetForMessage(ctx context.Context, dbConn db.ConnOrTx, msgID string) (*models.Snippet, error) {
iresult, err := db.QueryOne(ctx, dbConn, models.Snippet{},
snippet, err := db.QueryOne[models.Snippet](ctx, dbConn,
`
SELECT $columns
FROM handmade_snippet
FROM snippet
WHERE discord_message_id = $1
`,
msgID,
@ -669,7 +662,7 @@ func FetchSnippetForMessage(ctx context.Context, dbConn db.ConnOrTx, msgID strin
}
}
return iresult.(*models.Snippet), nil
return snippet, nil
}
/*
@ -718,7 +711,7 @@ func HandleSnippetForInternedMessage(ctx context.Context, dbConn db.ConnOrTx, in
_, err := tx.Exec(ctx,
`
UPDATE handmade_snippet
UPDATE snippet
SET
description = $1,
_description_html = $2
@ -747,7 +740,7 @@ func HandleSnippetForInternedMessage(ctx context.Context, dbConn db.ConnOrTx, in
_, err = tx.Exec(ctx,
`
INSERT INTO handmade_snippet (url, "when", description, _description_html, asset_id, discord_message_id, owner_id)
INSERT INTO snippet (url, "when", description, _description_html, asset_id, discord_message_id, owner_id)
VALUES ($1, $2, $3, $4, $5, $6, $7)
`,
url,
@ -769,7 +762,7 @@ func HandleSnippetForInternedMessage(ctx context.Context, dbConn db.ConnOrTx, in
_, err = tx.Exec(ctx,
`
UPDATE handmade_discordmessage
UPDATE discord_message
SET snippet_created = TRUE
WHERE id = $1
`,
@ -805,15 +798,12 @@ func HandleSnippetForInternedMessage(ctx context.Context, dbConn db.ConnOrTx, in
projectIDs[i] = p.Project.ID
}
type tagsRow struct {
Tag models.Tag `db:"tags"`
}
iUserTags, err := db.Query(ctx, tx, tagsRow{},
userTags, err := db.Query[models.Tag](ctx, tx,
`
SELECT $columns
SELECT $columns{tag}
FROM
tags
JOIN handmade_project AS project ON project.tag = tags.id
tag
JOIN project ON project.tag = tag.id
WHERE
project.id = ANY ($1)
`,
@ -823,8 +813,7 @@ func HandleSnippetForInternedMessage(ctx context.Context, dbConn db.ConnOrTx, in
return oops.New(err, "failed to fetch tags for user projects")
}
for _, itag := range iUserTags {
tag := itag.(*tagsRow).Tag
for _, tag := range userTags {
allTags = append(allTags, tag.ID)
for _, messageTag := range messageTags {
if strings.EqualFold(tag.Text, messageTag) {
@ -835,7 +824,7 @@ func HandleSnippetForInternedMessage(ctx context.Context, dbConn db.ConnOrTx, in
_, err = tx.Exec(ctx,
`
DELETE FROM snippet_tags
DELETE FROM snippet_tag
WHERE
snippet_id = $1
AND tag_id = ANY ($2)
@ -850,7 +839,7 @@ func HandleSnippetForInternedMessage(ctx context.Context, dbConn db.ConnOrTx, in
for _, tagID := range desiredTags {
_, err = tx.Exec(ctx,
`
INSERT INTO snippet_tags (snippet_id, tag_id)
INSERT INTO snippet_tag (snippet_id, tag_id)
VALUES ($1, $2)
ON CONFLICT DO NOTHING
`,
@ -890,10 +879,10 @@ var RESnippetableUrl = regexp.MustCompile(`^https?://(youtu\.be|(www\.)?youtube\
func getSnippetAssetOrUrl(ctx context.Context, tx db.ConnOrTx, msg *models.DiscordMessage) (*uuid.UUID, *string, error) {
// Check attachments
attachments, err := db.Query(ctx, tx, models.DiscordMessageAttachment{},
attachments, err := db.Query[models.DiscordMessageAttachment](ctx, tx,
`
SELECT $columns
FROM handmade_discordmessageattachment
FROM discord_message_attachment
WHERE message_id = $1
`,
msg.ID,
@ -901,16 +890,15 @@ func getSnippetAssetOrUrl(ctx context.Context, tx db.ConnOrTx, msg *models.Disco
if err != nil {
return nil, nil, oops.New(err, "failed to fetch message attachments")
}
for _, iattachment := range attachments {
attachment := iattachment.(*models.DiscordMessageAttachment)
for _, attachment := range attachments {
return &attachment.AssetID, nil, nil
}
// Check embeds
embeds, err := db.Query(ctx, tx, models.DiscordMessageEmbed{},
embeds, err := db.Query[models.DiscordMessageEmbed](ctx, tx,
`
SELECT $columns
FROM handmade_discordmessageembed
FROM discord_message_embed
WHERE message_id = $1
`,
msg.ID,
@ -918,8 +906,7 @@ func getSnippetAssetOrUrl(ctx context.Context, tx db.ConnOrTx, msg *models.Disco
if err != nil {
return nil, nil, oops.New(err, "failed to fetch discord embeds")
}
for _, iembed := range embeds {
embed := iembed.(*models.DiscordMessageEmbed)
for _, embed := range embeds {
if embed.VideoID != nil {
return embed.VideoID, nil, nil
} else if embed.ImageID != nil {

View File

@ -184,8 +184,26 @@ func GetGuildMember(ctx context.Context, guildID, userID string) (*GuildMember,
return &msg, nil
}
type MentionType string
const (
MentionTypeUsers MentionType = "users"
MentionTypeRoles = "roles"
MentionTypeEveryone = "everyone"
)
type MessageAllowedMentions struct {
Parse []MentionType `json:"parse"`
}
const (
FlagSuppressEmbeds int = 1 << 2
)
type CreateMessageRequest struct {
Content string `json:"content"`
Content string `json:"content"`
Flags int `json:"flags,omitempty"`
AllowedMentions *MessageAllowedMentions `json:"allowed_mentions,omitempty"`
}
func CreateMessage(ctx context.Context, channelID string, payloadJSON string, files ...FileUpload) (*Message, error) {
@ -226,6 +244,44 @@ func CreateMessage(ctx context.Context, channelID string, payloadJSON string, fi
return &msg, nil
}
func EditMessage(ctx context.Context, channelID string, messageID string, payloadJSON string, files ...FileUpload) (*Message, error) {
const name = "Edit Message"
contentType, body := makeNewMessageBody(payloadJSON, files)
path := fmt.Sprintf("/channels/%s/messages/%s", channelID, messageID)
res, err := doWithRateLimiting(ctx, name, func(ctx context.Context) *http.Request {
req := makeRequest(ctx, http.MethodPatch, path, body)
req.Header.Add("Content-Type", contentType)
return req
})
if err != nil {
return nil, err
}
defer res.Body.Close()
if res.StatusCode >= 400 {
logErrorResponse(ctx, name, res, "")
return nil, oops.New(nil, "received error from Discord")
}
// Maybe in the future we could more nicely handle errors like "bad channel",
// but honestly what are the odds that we mess that up...
bodyBytes, err := io.ReadAll(res.Body)
if err != nil {
panic(err)
}
var msg Message
err = json.Unmarshal(bodyBytes, &msg)
if err != nil {
return nil, oops.New(err, "failed to unmarshal Discord message")
}
return &msg, nil
}
func DeleteMessage(ctx context.Context, channelID string, messageID string) error {
const name = "Delete Message"

139
src/discord/streams.go Normal file
View File

@ -0,0 +1,139 @@
package discord
import (
"context"
"encoding/json"
"fmt"
"strings"
"git.handmade.network/hmn/hmn/src/config"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/hmndata"
"git.handmade.network/hmn/hmn/src/oops"
)
// NOTE(asaf): Updates or creates a discord message according to the following rules:
// Create when:
// * No previous message exists
// * We have non-zero live streamers
// * Message exists, but we're adding a new streamer that wasn't in the existing message
// * Message exists, but is not the most recent message in the channel
// Update otherwise
// That way we ensure that the message doesn't get scrolled offscreen, and the
// new message indicator for the channel doesn't trigger when a streamer goes offline or
// updates the stream title.
// NOTE(asaf): No-op if StreamsChannelID is not specified in the config
func UpdateStreamers(ctx context.Context, dbConn db.ConnOrTx, streamers []hmndata.StreamDetails) error {
if len(config.Config.Discord.StreamsChannelID) == 0 {
return nil
}
livestreamMessage, err := hmndata.FetchPersistentVar[hmndata.DiscordLivestreamMessage](
ctx,
dbConn,
hmndata.VarNameDiscordLivestreamMessage,
)
editExisting := true
if err != nil {
if err == db.NotFound {
editExisting = false
} else {
return oops.New(err, "failed to fetch last message persistent var from db")
}
}
if editExisting {
_, err := GetChannelMessage(ctx, config.Config.Discord.StreamsChannelID, livestreamMessage.MessageID)
if err != nil {
if err == NotFound {
editExisting = false
} else {
oops.New(err, "failed to fetch existing message from discord")
}
}
}
if editExisting {
existingStreamers := livestreamMessage.Streamers
for _, s := range streamers {
found := false
for _, es := range existingStreamers {
if es.Username == s.Username {
found = true
break
}
}
if !found {
editExisting = false
break
}
}
}
if editExisting && len(streamers) > 0 {
messages, err := GetChannelMessages(ctx, config.Config.Discord.StreamsChannelID, GetChannelMessagesInput{
Limit: 1,
})
if err != nil {
return oops.New(err, "failed to fetch messages from discord")
}
if len(messages) == 0 || messages[0].ID != livestreamMessage.MessageID {
editExisting = false
}
}
messageContent := ""
if len(streamers) == 0 {
messageContent = "No one is currently streaming."
} else {
var builder strings.Builder
for _, s := range streamers {
builder.WriteString(fmt.Sprintf(":red_circle: **%s** is live: <https://twitch.tv/%s>\n> _%s_\nStarted <t:%d:R>\n\n", s.Username, s.Username, s.Title, s.StartTime.Unix()))
}
messageContent = builder.String()
}
msgJson, err := json.Marshal(CreateMessageRequest{
Content: messageContent,
Flags: FlagSuppressEmbeds,
AllowedMentions: &MessageAllowedMentions{},
})
if err != nil {
return oops.New(err, "failed to marshal discord message")
}
newMessageID := ""
if editExisting {
updatedMessage, err := EditMessage(ctx, config.Config.Discord.StreamsChannelID, livestreamMessage.MessageID, string(msgJson))
if err != nil {
return oops.New(err, "failed to update discord message for streams channel")
}
newMessageID = updatedMessage.ID
} else {
if livestreamMessage != nil {
err = DeleteMessage(ctx, config.Config.Discord.StreamsChannelID, livestreamMessage.MessageID)
if err != nil {
return oops.New(err, "failed to delete existing discord message from streams channel")
}
}
sentMessage, err := CreateMessage(ctx, config.Config.Discord.StreamsChannelID, string(msgJson))
if err != nil {
return oops.New(err, "failed to create discord message for streams channel")
}
newMessageID = sentMessage.ID
}
data := hmndata.DiscordLivestreamMessage{
MessageID: newMessageID,
Streamers: streamers,
}
err = hmndata.StorePersistentVar(ctx, dbConn, hmndata.VarNameDiscordLivestreamMessage, &data)
if err != nil {
return oops.New(err, "failed to store persistent var for discord streams")
}
return nil
}

View File

@ -106,8 +106,8 @@ func renderTemplate(name string, data interface{}) (string, error) {
}
func sendMail(toAddress, toName, subject, contentHtml string) error {
if config.Config.Email.OverrideRecipientEmail != "" {
toAddress = config.Config.Email.OverrideRecipientEmail
if config.Config.Email.ForceToAddress != "" {
toAddress = config.Config.Email.ForceToAddress
}
contents := prepMailContents(
makeHeaderAddress(toAddress, toName),

View File

@ -0,0 +1,86 @@
package hmndata
import (
"context"
"encoding/json"
"time"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/models"
"git.handmade.network/hmn/hmn/src/oops"
)
type PersistentVarName string
const (
VarNameDiscordLivestreamMessage PersistentVarName = "discord_livestream_message"
)
type StreamDetails struct {
Username string `json:"username"`
StartTime time.Time `json:"start_time"`
Title string `json:"title"`
}
type DiscordLivestreamMessage struct {
MessageID string `json:"message_id"`
Streamers []StreamDetails `json:"streamers"`
}
// NOTE(asaf): Returns db.NotFound if the variable isn't in the db.
func FetchPersistentVar[T any](
ctx context.Context,
dbConn db.ConnOrTx,
varName PersistentVarName,
) (*T, error) {
persistentVar, err := db.QueryOne[models.PersistentVar](ctx, dbConn,
`
SELECT $columns
FROM persistent_var
WHERE name = $1
`,
varName,
)
if err != nil {
return nil, err
}
jsonString := persistentVar.Value
var result T
err = json.Unmarshal([]byte(jsonString), &result)
if err != nil {
return nil, oops.New(err, "failed to unmarshal persistent var value")
}
return &result, nil
}
func StorePersistentVar[T any](
ctx context.Context,
dbConn db.ConnOrTx,
name PersistentVarName,
value *T,
) error {
jsonString, err := json.Marshal(value)
if err != nil {
return oops.New(err, "failed to marshal variable")
}
_, err = dbConn.Exec(ctx,
`
INSERT INTO persistent_var (name, value)
VALUES ($1, $2)
ON CONFLICT (name) DO UPDATE SET
value = EXCLUDED.value
`,
name,
jsonString,
)
if err != nil {
return oops.New(err, "failed to insert var to db")
}
return nil
}

View File

@ -71,7 +71,7 @@ func FetchProjects(
Project models.Project `db:"project"`
LogoLightAsset *models.Asset `db:"logolight_asset"`
LogoDarkAsset *models.Asset `db:"logodark_asset"`
Tag *models.Tag `db:"tags"`
Tag *models.Tag `db:"tag"`
}
// Fetch all valid projects (not yet subject to user permission checks)
@ -82,17 +82,17 @@ func FetchProjects(
qb.Add(`
SELECT DISTINCT ON (project.id) $columns
FROM
handmade_project AS project
LEFT JOIN handmade_asset AS logolight_asset ON logolight_asset.id = project.logolight_asset_id
LEFT JOIN handmade_asset AS logodark_asset ON logodark_asset.id = project.logodark_asset_id
LEFT JOIN tags ON project.tag = tags.id
project
LEFT JOIN asset AS logolight_asset ON logolight_asset.id = project.logolight_asset_id
LEFT JOIN asset AS logodark_asset ON logodark_asset.id = project.logodark_asset_id
LEFT JOIN tag ON project.tag = tag.id
`)
if len(q.OwnerIDs) > 0 {
qb.Add(
`
JOIN (
SELECT project_id, array_agg(user_id) AS owner_ids
FROM handmade_user_projects
FROM user_project
WHERE user_id = ANY ($?)
GROUP BY project_id
) AS owner_filter ON project.id = owner_filter.project_id
@ -140,15 +140,15 @@ func FetchProjects(
}
// Do the query
iprojects, err := db.Query(ctx, dbConn, projectRow{}, qb.String(), qb.Args()...)
projectRows, err := db.Query[projectRow](ctx, dbConn, qb.String(), qb.Args()...)
if err != nil {
return nil, oops.New(err, "failed to fetch projects")
}
// Fetch project owners to do permission checks
projectIds := make([]int, len(iprojects))
for i, iproject := range iprojects {
projectIds[i] = iproject.(*projectRow).Project.ID
projectIds := make([]int, len(projectRows))
for i, p := range projectRows {
projectIds[i] = p.Project.ID
}
projectOwners, err := FetchMultipleProjectsOwners(ctx, tx, projectIds)
if err != nil {
@ -156,8 +156,7 @@ func FetchProjects(
}
var res []ProjectAndStuff
for i, iproject := range iprojects {
row := iproject.(*projectRow)
for i, p := range projectRows {
owners := projectOwners[i].Owners
/*
@ -191,10 +190,10 @@ func FetchProjects(
}
projectGenerallyVisible := true &&
row.Project.Lifecycle.In(models.VisibleProjectLifecycles) &&
!row.Project.Hidden &&
(!row.Project.Personal || allOwnersApproved || row.Project.IsHMN())
if row.Project.IsHMN() {
p.Project.Lifecycle.In(models.VisibleProjectLifecycles) &&
!p.Project.Hidden &&
(!p.Project.Personal || allOwnersApproved || p.Project.IsHMN())
if p.Project.IsHMN() {
projectGenerallyVisible = true // hard override
}
@ -205,11 +204,11 @@ func FetchProjects(
if projectVisible {
res = append(res, ProjectAndStuff{
Project: row.Project,
LogoLightAsset: row.LogoLightAsset,
LogoDarkAsset: row.LogoDarkAsset,
Project: p.Project,
LogoLightAsset: p.LogoLightAsset,
LogoDarkAsset: p.LogoDarkAsset,
Owners: owners,
Tag: row.Tag,
Tag: p.Tag,
})
}
}
@ -334,10 +333,10 @@ func FetchMultipleProjectsOwners(
UserID int `db:"user_id"`
ProjectID int `db:"project_id"`
}
iuserprojects, err := db.Query(ctx, tx, userProject{},
userProjects, err := db.Query[userProject](ctx, tx,
`
SELECT $columns
FROM handmade_user_projects
FROM user_project
WHERE project_id = ANY($1)
`,
projectIds,
@ -348,9 +347,7 @@ func FetchMultipleProjectsOwners(
// Get the unique user IDs from this set and fetch the users from the db
var userIds []int
for _, iuserproject := range iuserprojects {
userProject := iuserproject.(*userProject)
for _, userProject := range userProjects {
addUserId := true
for _, uid := range userIds {
if uid == userProject.UserID {
@ -361,19 +358,10 @@ func FetchMultipleProjectsOwners(
userIds = append(userIds, userProject.UserID)
}
}
type userQuery struct {
User models.User `db:"auth_user"`
}
iusers, err := db.Query(ctx, tx, userQuery{},
`
SELECT $columns
FROM auth_user
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
WHERE
auth_user.id = ANY($1)
`,
userIds,
)
users, err := FetchUsers(ctx, tx, nil, UsersQuery{
UserIDs: userIds,
AnyStatus: true,
})
if err != nil {
return nil, oops.New(err, "failed to fetch users for projects")
}
@ -383,9 +371,7 @@ func FetchMultipleProjectsOwners(
for i, pid := range projectIds {
res[i] = ProjectOwners{ProjectID: pid}
}
for _, iuserproject := range iuserprojects {
userProject := iuserproject.(*userProject)
for _, userProject := range userProjects {
// Get a pointer to the existing record in the result
var projectOwners *ProjectOwners
for i := range res {
@ -396,10 +382,9 @@ func FetchMultipleProjectsOwners(
// Get the full user record we fetched
var user *models.User
for _, iuser := range iusers {
u := iuser.(*userQuery).User
for _, u := range users {
if u.ID == userProject.UserID {
user = &u
user = u
}
}
if user == nil {
@ -473,9 +458,9 @@ func SetProjectTag(
resultTag = p.Tag
} else if p.Project.TagID == nil {
// Create a tag
itag, err := db.QueryOne(ctx, tx, models.Tag{},
tag, err := db.QueryOne[models.Tag](ctx, tx,
`
INSERT INTO tags (text) VALUES ($1)
INSERT INTO tag (text) VALUES ($1)
RETURNING $columns
`,
tagText,
@ -483,12 +468,12 @@ func SetProjectTag(
if err != nil {
return nil, oops.New(err, "failed to create new tag for project")
}
resultTag = itag.(*models.Tag)
resultTag = tag
// Attach it to the project
_, err = tx.Exec(ctx,
`
UPDATE handmade_project
UPDATE project
SET tag = $1
WHERE id = $2
`,
@ -499,11 +484,11 @@ func SetProjectTag(
}
} else {
// Update the text of an existing one
itag, err := db.QueryOne(ctx, tx, models.Tag{},
tag, err := db.QueryOne[models.Tag](ctx, tx,
`
UPDATE tags
UPDATE tag
SET text = $1
WHERE id = (SELECT tag FROM handmade_project WHERE id = $2)
WHERE id = (SELECT tag FROM project WHERE id = $2)
RETURNING $columns
`,
tagText, projectID,
@ -511,7 +496,7 @@ func SetProjectTag(
if err != nil {
return nil, oops.New(err, "failed to update existing tag")
}
resultTag = itag.(*models.Tag)
resultTag = tag
}
err = tx.Commit(ctx)

View File

@ -44,17 +44,14 @@ func FetchSnippets(
if len(q.Tags) > 0 {
// Get snippet IDs with this tag, then use that in the main query
type snippetIDRow struct {
SnippetID int `db:"snippet_id"`
}
iSnippetIDs, err := db.Query(ctx, tx, snippetIDRow{},
snippetIDs, err := db.QueryScalar[int](ctx, tx,
`
SELECT DISTINCT snippet_id
FROM
snippet_tags
JOIN tags ON snippet_tags.tag_id = tags.id
snippet_tag
JOIN tag ON snippet_tag.tag_id = tag.id
WHERE
tags.id = ANY ($1)
tag.id = ANY ($1)
`,
q.Tags,
)
@ -63,14 +60,11 @@ func FetchSnippets(
}
// special early-out: no snippets found for these tags at all
if len(iSnippetIDs) == 0 {
if len(snippetIDs) == 0 {
return nil, nil
}
q.IDs = make([]int, len(iSnippetIDs))
for i := range iSnippetIDs {
q.IDs[i] = iSnippetIDs[i].(*snippetIDRow).SnippetID
}
q.IDs = snippetIDs
}
var qb db.QueryBuilder
@ -78,11 +72,11 @@ func FetchSnippets(
`
SELECT $columns
FROM
handmade_snippet AS snippet
LEFT JOIN auth_user AS owner ON snippet.owner_id = owner.id
LEFT JOIN handmade_asset AS owner_avatar ON owner_avatar.id = owner.avatar_asset_id
LEFT JOIN handmade_asset AS asset ON snippet.asset_id = asset.id
LEFT JOIN handmade_discordmessage AS discord_message ON snippet.discord_message_id = discord_message.id
snippet
LEFT JOIN hmn_user AS owner ON snippet.owner_id = owner.id
LEFT JOIN asset AS owner_avatar ON owner_avatar.id = owner.avatar_asset_id
LEFT JOIN asset ON snippet.asset_id = asset.id
LEFT JOIN discord_message ON snippet.discord_message_id = discord_message.id
WHERE
TRUE
`,
@ -125,16 +119,14 @@ func FetchSnippets(
DiscordMessage *models.DiscordMessage `db:"discord_message"`
}
iresults, err := db.Query(ctx, tx, resultRow{}, qb.String(), qb.Args()...)
results, err := db.Query[resultRow](ctx, tx, qb.String(), qb.Args()...)
if err != nil {
return nil, oops.New(err, "failed to fetch threads")
}
result := make([]SnippetAndStuff, len(iresults)) // allocate extra space because why not
snippetIDs := make([]int, len(iresults))
for i, iresult := range iresults {
row := *iresult.(*resultRow)
result := make([]SnippetAndStuff, len(results)) // allocate extra space because why not
snippetIDs := make([]int, len(results))
for i, row := range results {
result[i] = SnippetAndStuff{
Snippet: row.Snippet,
Owner: row.Owner,
@ -147,17 +139,17 @@ func FetchSnippets(
// Fetch tags
type snippetTagRow struct {
SnippetID int `db:"snippet_tags.snippet_id"`
Tag *models.Tag `db:"tags"`
SnippetID int `db:"snippet_tag.snippet_id"`
Tag *models.Tag `db:"tag"`
}
iSnippetTags, err := db.Query(ctx, tx, snippetTagRow{},
snippetTags, err := db.Query[snippetTagRow](ctx, tx,
`
SELECT $columns
FROM
snippet_tags
JOIN tags ON snippet_tags.tag_id = tags.id
snippet_tag
JOIN tag ON snippet_tag.tag_id = tag.id
WHERE
snippet_tags.snippet_id = ANY($1)
snippet_tag.snippet_id = ANY($1)
`,
snippetIDs,
)
@ -170,8 +162,7 @@ func FetchSnippets(
for i := range result {
resultBySnippetId[result[i].Snippet.ID] = &result[i]
}
for _, iSnippetTag := range iSnippetTags {
snippetTag := iSnippetTag.(*snippetTagRow)
for _, snippetTag := range snippetTags {
item := resultBySnippetId[snippetTag.SnippetID]
item.Tags = append(item.Tags, snippetTag.Tag)
}

View File

@ -25,7 +25,7 @@ func FetchTags(ctx context.Context, dbConn db.ConnOrTx, q TagQuery) ([]*models.T
qb.Add(
`
SELECT $columns
FROM tags
FROM tag
WHERE
TRUE
`,
@ -40,18 +40,11 @@ func FetchTags(ctx context.Context, dbConn db.ConnOrTx, q TagQuery) ([]*models.T
qb.Add(`LIMIT $? OFFSET $?`, q.Limit, q.Offset)
}
itags, err := db.Query(ctx, dbConn, models.Tag{}, qb.String(), qb.Args()...)
tags, err := db.Query[models.Tag](ctx, dbConn, qb.String(), qb.Args()...)
if err != nil {
return nil, oops.New(err, "failed to fetch tags")
}
res := make([]*models.Tag, len(itags))
for i, itag := range itags {
tag := itag.(*models.Tag)
res[i] = tag
}
return res, nil
return tags, nil
}
func FetchTag(ctx context.Context, dbConn db.ConnOrTx, q TagQuery) (*models.Tag, error) {

View File

@ -71,21 +71,21 @@ func FetchThreads(
`
SELECT $columns
FROM
handmade_thread AS thread
JOIN handmade_project AS project ON thread.project_id = project.id
JOIN handmade_post AS first_post ON first_post.id = thread.first_id
JOIN handmade_post AS last_post ON last_post.id = thread.last_id
JOIN handmade_postversion AS first_version ON first_version.id = first_post.current_id
JOIN handmade_postversion AS last_version ON last_version.id = last_post.current_id
LEFT JOIN auth_user AS first_author ON first_author.id = first_post.author_id
LEFT JOIN handmade_asset AS first_author_avatar ON first_author_avatar.id = first_author.avatar_asset_id
LEFT JOIN auth_user AS last_author ON last_author.id = last_post.author_id
LEFT JOIN handmade_asset AS last_author_avatar ON last_author_avatar.id = last_author.avatar_asset_id
LEFT JOIN handmade_threadlastreadinfo AS tlri ON (
thread
JOIN project ON thread.project_id = project.id
JOIN post AS first_post ON first_post.id = thread.first_id
JOIN post AS last_post ON last_post.id = thread.last_id
JOIN post_version AS first_version ON first_version.id = first_post.current_id
JOIN post_version AS last_version ON last_version.id = last_post.current_id
LEFT JOIN hmn_user AS first_author ON first_author.id = first_post.author_id
LEFT JOIN asset AS first_author_avatar ON first_author_avatar.id = first_author.avatar_asset_id
LEFT JOIN hmn_user AS last_author ON last_author.id = last_post.author_id
LEFT JOIN asset AS last_author_avatar ON last_author_avatar.id = last_author.avatar_asset_id
LEFT JOIN thread_last_read_info AS tlri ON (
tlri.thread_id = thread.id
AND tlri.user_id = $?
)
LEFT JOIN handmade_subforumlastreadinfo AS slri ON (
LEFT JOIN subforum_last_read_info AS slri ON (
slri.subforum_id = thread.subforum_id
AND slri.user_id = $?
)
@ -141,18 +141,25 @@ func FetchThreads(
type resultRow struct {
ThreadAndStuff
ThreadLastReadTime *time.Time `db:"tlri.lastread"`
ForumLastReadTime *time.Time `db:"slri.lastread"`
FirstPostAuthorAvatar *models.Asset `db:"first_author_avatar"`
LastPostAuthorAvatar *models.Asset `db:"last_author_avatar"`
ThreadLastReadTime *time.Time `db:"tlri.lastread"`
ForumLastReadTime *time.Time `db:"slri.lastread"`
}
iresults, err := db.Query(ctx, dbConn, resultRow{}, qb.String(), qb.Args()...)
rows, err := db.Query[resultRow](ctx, dbConn, qb.String(), qb.Args()...)
if err != nil {
return nil, oops.New(err, "failed to fetch threads")
}
result := make([]ThreadAndStuff, len(iresults))
for i, iresult := range iresults {
row := *iresult.(*resultRow)
result := make([]ThreadAndStuff, len(rows))
for i, row := range rows {
if row.FirstPostAuthor != nil {
row.FirstPostAuthor.AvatarAsset = row.FirstPostAuthorAvatar
}
if row.LastPostAuthor != nil {
row.LastPostAuthor.AvatarAsset = row.LastPostAuthorAvatar
}
hasRead := false
if currentUser != nil && currentUser.MarkedAllReadAt.After(row.LastPost.PostDate) {
@ -221,11 +228,11 @@ func CountThreads(
`
SELECT COUNT(*)
FROM
handmade_thread AS thread
JOIN handmade_project AS project ON thread.project_id = project.id
JOIN handmade_post AS first_post ON first_post.id = thread.first_id
LEFT JOIN auth_user AS first_author ON first_author.id = first_post.author_id
LEFT JOIN handmade_asset AS first_author_avatar ON first_author_avatar.id = first_author.avatar_asset_id
thread
JOIN project ON thread.project_id = project.id
JOIN post AS first_post ON first_post.id = thread.first_id
LEFT JOIN hmn_user AS first_author ON first_author.id = first_post.author_id
LEFT JOIN asset AS first_author_avatar ON first_author_avatar.id = first_author.avatar_asset_id
WHERE
NOT thread.deleted
AND ( -- project has valid lifecycle
@ -263,7 +270,7 @@ func CountThreads(
)
}
count, err := db.QueryInt(ctx, dbConn, qb.String(), qb.Args()...)
count, err := db.QueryOneScalar[int](ctx, dbConn, qb.String(), qb.Args()...)
if err != nil {
return 0, oops.New(err, "failed to fetch count of threads")
}
@ -322,36 +329,39 @@ func FetchPosts(
type resultRow struct {
PostAndStuff
ThreadLastReadTime *time.Time `db:"tlri.lastread"`
ForumLastReadTime *time.Time `db:"slri.lastread"`
AuthorAvatar *models.Asset `db:"author_avatar"`
EditorAvatar *models.Asset `db:"editor_avatar"`
ReplyAuthorAvatar *models.Asset `db:"reply_author_avatar"`
ThreadLastReadTime *time.Time `db:"tlri.lastread"`
ForumLastReadTime *time.Time `db:"slri.lastread"`
}
qb.Add(
`
SELECT $columns
FROM
handmade_post AS post
JOIN handmade_thread AS thread ON post.thread_id = thread.id
JOIN handmade_project AS project ON post.project_id = project.id
JOIN handmade_postversion AS ver ON ver.id = post.current_id
LEFT JOIN auth_user AS author ON author.id = post.author_id
LEFT JOIN handmade_asset AS author_avatar ON author_avatar.id = author.avatar_asset_id
LEFT JOIN auth_user AS editor ON ver.editor_id = editor.id
LEFT JOIN handmade_asset AS editor_avatar ON editor_avatar.id = editor.avatar_asset_id
LEFT JOIN handmade_threadlastreadinfo AS tlri ON (
post
JOIN thread ON post.thread_id = thread.id
JOIN project ON post.project_id = project.id
JOIN post_version AS ver ON ver.id = post.current_id
LEFT JOIN hmn_user AS author ON author.id = post.author_id
LEFT JOIN asset AS author_avatar ON author_avatar.id = author.avatar_asset_id
LEFT JOIN hmn_user AS editor ON ver.editor_id = editor.id
LEFT JOIN asset AS editor_avatar ON editor_avatar.id = editor.avatar_asset_id
LEFT JOIN thread_last_read_info AS tlri ON (
tlri.thread_id = thread.id
AND tlri.user_id = $?
)
LEFT JOIN handmade_subforumlastreadinfo AS slri ON (
LEFT JOIN subforum_last_read_info AS slri ON (
slri.subforum_id = thread.subforum_id
AND slri.user_id = $?
)
-- Unconditionally fetch reply info, but make sure to check it
-- later and possibly remove these fields if the permission
-- check fails.
LEFT JOIN handmade_post AS reply_post ON reply_post.id = post.reply_id
LEFT JOIN auth_user AS reply_author ON reply_post.author_id = reply_author.id
LEFT JOIN handmade_asset AS reply_author_avatar ON reply_author_avatar.id = reply_author.avatar_asset_id
LEFT JOIN post AS reply_post ON reply_post.id = post.reply_id
LEFT JOIN hmn_user AS reply_author ON reply_post.author_id = reply_author.id
LEFT JOIN asset AS reply_author_avatar ON reply_author_avatar.id = reply_author.avatar_asset_id
WHERE
NOT thread.deleted
AND NOT post.deleted
@ -405,14 +415,22 @@ func FetchPosts(
qb.Add(`LIMIT $? OFFSET $?`, q.Limit, q.Offset)
}
iresults, err := db.Query(ctx, dbConn, resultRow{}, qb.String(), qb.Args()...)
rows, err := db.Query[resultRow](ctx, dbConn, qb.String(), qb.Args()...)
if err != nil {
return nil, oops.New(err, "failed to fetch posts")
}
result := make([]PostAndStuff, len(iresults))
for i, iresult := range iresults {
row := *iresult.(*resultRow)
result := make([]PostAndStuff, len(rows))
for i, row := range rows {
if row.Author != nil {
row.Author.AvatarAsset = row.AuthorAvatar
}
if row.Editor != nil {
row.Editor.AvatarAsset = row.EditorAvatar
}
if row.ReplyAuthor != nil {
row.ReplyAuthor.AvatarAsset = row.ReplyAuthorAvatar
}
hasRead := false
if currentUser != nil && currentUser.MarkedAllReadAt.After(row.Post.PostDate) {
@ -549,11 +567,11 @@ func CountPosts(
`
SELECT COUNT(*)
FROM
handmade_post AS post
JOIN handmade_thread AS thread ON post.thread_id = thread.id
JOIN handmade_project AS project ON post.project_id = project.id
LEFT JOIN auth_user AS author ON author.id = post.author_id
LEFT JOIN handmade_asset AS author_avatar ON author_avatar.id = author.avatar_asset_id
post
JOIN thread ON post.thread_id = thread.id
JOIN project ON post.project_id = project.id
LEFT JOIN hmn_user AS author ON author.id = post.author_id
LEFT JOIN asset AS author_avatar ON author_avatar.id = author.avatar_asset_id
WHERE
NOT thread.deleted
AND NOT post.deleted
@ -595,7 +613,7 @@ func CountPosts(
)
}
count, err := db.QueryInt(ctx, dbConn, qb.String(), qb.Args()...)
count, err := db.QueryOneScalar[int](ctx, dbConn, qb.String(), qb.Args()...)
if err != nil {
return 0, oops.New(err, "failed to count posts")
}
@ -608,14 +626,11 @@ func UserCanEditPost(ctx context.Context, connOrTx db.ConnOrTx, user models.User
return true
}
type postResult struct {
AuthorID *int `db:"post.author_id"`
}
iresult, err := db.QueryOne(ctx, connOrTx, postResult{},
authorID, err := db.QueryOneScalar[*int](ctx, connOrTx,
`
SELECT $columns
SELECT post.author_id
FROM
handmade_post AS post
post
WHERE
post.id = $1
AND NOT post.deleted
@ -629,9 +644,8 @@ func UserCanEditPost(ctx context.Context, connOrTx db.ConnOrTx, user models.User
panic(oops.New(err, "failed to get author of post when checking permissions"))
}
}
result := iresult.(*postResult)
return result.AuthorID != nil && *result.AuthorID == user.ID
return authorID != nil && *authorID == user.ID
}
func CreateNewPost(
@ -647,7 +661,7 @@ func CreateNewPost(
// Create post
err := tx.QueryRow(ctx,
`
INSERT INTO handmade_post (postdate, thread_id, thread_type, current_id, author_id, project_id, reply_id, preview)
INSERT INTO post (postdate, thread_id, thread_type, current_id, author_id, project_id, reply_id, preview)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING id
`,
@ -688,7 +702,7 @@ func CreateNewPost(
_, err = tx.Exec(ctx,
`
UPDATE handmade_project
UPDATE project
SET `+updates+`
WHERE
id = $1
@ -709,11 +723,11 @@ func DeletePost(
FirstPostID int `db:"first_id"`
Deleted bool `db:"deleted"`
}
ti, err := db.QueryOne(ctx, tx, threadInfo{},
info, err := db.QueryOne[threadInfo](ctx, tx,
`
SELECT $columns
FROM
handmade_thread AS thread
thread
WHERE
thread.id = $1
`,
@ -722,7 +736,6 @@ func DeletePost(
if err != nil {
panic(oops.New(err, "failed to fetch thread info"))
}
info := ti.(*threadInfo)
if info.Deleted {
return true
}
@ -732,7 +745,7 @@ func DeletePost(
// Just delete the whole thread and all its posts.
_, err = tx.Exec(ctx,
`
UPDATE handmade_thread
UPDATE thread
SET deleted = TRUE
WHERE id = $1
`,
@ -740,7 +753,7 @@ func DeletePost(
)
_, err = tx.Exec(ctx,
`
UPDATE handmade_post
UPDATE post
SET deleted = TRUE
WHERE thread_id = $1
`,
@ -752,7 +765,7 @@ func DeletePost(
_, err = tx.Exec(ctx,
`
UPDATE handmade_post
UPDATE post
SET deleted = TRUE
WHERE
id = $1
@ -798,7 +811,7 @@ func CreatePostVersion(ctx context.Context, tx pgx.Tx, postId int, unparsedConte
// Create post version
err := tx.QueryRow(ctx,
`
INSERT INTO handmade_postversion (post_id, text_raw, text_parsed, ip, date, edit_reason, editor_id)
INSERT INTO post_version (post_id, text_raw, text_parsed, ip, date, edit_reason, editor_id)
VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING id
`,
@ -817,7 +830,7 @@ func CreatePostVersion(ctx context.Context, tx pgx.Tx, postId int, unparsedConte
// Update post with version id and preview
_, err = tx.Exec(ctx,
`
UPDATE handmade_post
UPDATE post
SET current_id = $1, preview = $2
WHERE id = $3
`,
@ -833,7 +846,7 @@ func CreatePostVersion(ctx context.Context, tx pgx.Tx, postId int, unparsedConte
_, err = tx.Exec(ctx,
`
DELETE FROM handmade_post_asset_usage
DELETE FROM post_asset_usage
WHERE post_id = $1
`,
postId,
@ -848,13 +861,10 @@ func CreatePostVersion(ctx context.Context, tx pgx.Tx, postId int, unparsedConte
keys = append(keys, key)
}
type assetId struct {
AssetID uuid.UUID `db:"id"`
}
assetResult, err := db.Query(ctx, tx, assetId{},
assetIDs, err := db.QueryScalar[uuid.UUID](ctx, tx,
`
SELECT $columns
FROM handmade_asset
SELECT id
FROM asset
WHERE s3_key = ANY($1)
`,
keys,
@ -865,11 +875,11 @@ func CreatePostVersion(ctx context.Context, tx pgx.Tx, postId int, unparsedConte
var values [][]interface{}
for _, asset := range assetResult {
values = append(values, []interface{}{postId, asset.(*assetId).AssetID})
for _, assetID := range assetIDs {
values = append(values, []interface{}{postId, assetID})
}
_, err = tx.CopyFrom(ctx, pgx.Identifier{"handmade_post_asset_usage"}, []string{"post_id", "asset_id"}, pgx.CopyFromRows(values))
_, err = tx.CopyFrom(ctx, pgx.Identifier{"post_asset_usage"}, []string{"post_id", "asset_id"}, pgx.CopyFromRows(values))
if err != nil {
panic(oops.New(err, "failed to insert post asset usage"))
}
@ -885,11 +895,11 @@ Ensures that the first_id and last_id on the thread are still good.
Returns errThreadEmpty if the thread contains no visible posts any more.
You should probably mark the thread as deleted in this case.
*/
func FixThreadPostIds(ctx context.Context, tx pgx.Tx, threadId int) error {
postsIter, err := db.Query(ctx, tx, models.Post{},
func FixThreadPostIds(ctx context.Context, conn db.ConnOrTx, threadId int) error {
posts, err := db.Query[models.Post](ctx, conn,
`
SELECT $columns
FROM handmade_post
FROM post
WHERE
thread_id = $1
AND NOT deleted
@ -901,9 +911,7 @@ func FixThreadPostIds(ctx context.Context, tx pgx.Tx, threadId int) error {
}
var firstPost, lastPost *models.Post
for _, ipost := range postsIter {
post := ipost.(*models.Post)
for _, post := range posts {
if firstPost == nil || post.PostDate.Before(firstPost.PostDate) {
firstPost = post
}
@ -916,9 +924,9 @@ func FixThreadPostIds(ctx context.Context, tx pgx.Tx, threadId int) error {
return errThreadEmpty
}
_, err = tx.Exec(ctx,
_, err = conn.Exec(ctx,
`
UPDATE handmade_thread
UPDATE thread
SET first_id = $1, last_id = $2
WHERE id = $3
`,

View File

@ -22,22 +22,19 @@ type TwitchStreamer struct {
var twitchRegex = regexp.MustCompile(`twitch\.tv/(?P<login>[^/]+)$`)
func FetchTwitchStreamers(ctx context.Context, dbConn db.ConnOrTx) ([]TwitchStreamer, error) {
type linkResult struct {
Link models.Link `db:"link"`
}
streamers, err := db.Query(ctx, dbConn, linkResult{},
dbStreamers, err := db.Query[models.Link](ctx, dbConn,
`
SELECT $columns
SELECT $columns{link}
FROM
handmade_links AS link
LEFT JOIN auth_user AS link_owner ON link_owner.id = link.user_id
link
LEFT JOIN hmn_user AS link_owner ON link_owner.id = link.user_id
WHERE
url ~* 'twitch\.tv/([^/]+)$' AND
((link.user_id IS NOT NULL AND link_owner.status = $1) OR (link.project_id IS NOT NULL AND
(SELECT COUNT(*)
FROM
handmade_user_projects AS hup
JOIN auth_user AS project_owner ON project_owner.id = hup.user_id
user_project AS hup
JOIN hmn_user AS project_owner ON project_owner.id = hup.user_id
WHERE
hup.project_id = link.project_id AND
project_owner.status != $1
@ -49,10 +46,8 @@ func FetchTwitchStreamers(ctx context.Context, dbConn db.ConnOrTx) ([]TwitchStre
return nil, oops.New(err, "failed to fetch twitch links")
}
result := make([]TwitchStreamer, 0, len(streamers))
for _, s := range streamers {
dbStreamer := s.(*linkResult).Link
result := make([]TwitchStreamer, 0, len(dbStreamers))
for _, dbStreamer := range dbStreamers {
streamer := TwitchStreamer{
UserID: dbStreamer.UserID,
ProjectID: dbStreamer.ProjectID,
@ -81,11 +76,11 @@ func FetchTwitchStreamers(ctx context.Context, dbConn db.ConnOrTx) ([]TwitchStre
}
func FetchTwitchLoginsForUserOrProject(ctx context.Context, dbConn db.ConnOrTx, userId *int, projectId *int) ([]string, error) {
links, err := db.Query(ctx, dbConn, models.Link{},
links, err := db.Query[models.Link](ctx, dbConn,
`
SELECT $columns
FROM
handmade_links AS link
link
WHERE
url ~* 'twitch\.tv/([^/]+)$'
AND ((user_id = $1 AND project_id IS NULL) OR (user_id IS NULL AND project_id = $2))
@ -100,8 +95,7 @@ func FetchTwitchLoginsForUserOrProject(ctx context.Context, dbConn db.ConnOrTx,
result := make([]string, 0, len(links))
for _, l := range links {
url := l.(*models.Link).URL
match := twitchRegex.FindStringSubmatch(url)
match := twitchRegex.FindStringSubmatch(l.URL)
if match != nil {
login := strings.ToLower(match[twitchRegex.SubexpIndex("login")])
result = append(result, login)

154
src/hmndata/user_helper.go Normal file
View File

@ -0,0 +1,154 @@
package hmndata
import (
"context"
"strings"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/models"
"git.handmade.network/hmn/hmn/src/oops"
"git.handmade.network/hmn/hmn/src/perf"
)
type UsersQuery struct {
// Ignored when using FetchUser
UserIDs []int // if empty, all users
Usernames []string // if empty, all users
// Flags to modify behavior
AnyStatus bool // Bypasses shadowban system
}
/*
Fetches users and related models from the database according to all the given
query params. For the most correct results, provide as much information as you have
on hand.
*/
func FetchUsers(
ctx context.Context,
dbConn db.ConnOrTx,
currentUser *models.User,
q UsersQuery,
) ([]*models.User, error) {
perf := perf.ExtractPerf(ctx)
perf.StartBlock("SQL", "Fetch users")
defer perf.EndBlock()
var currentUserID *int
if currentUser != nil {
currentUserID = &currentUser.ID
}
for i := range q.Usernames {
q.Usernames[i] = strings.ToLower(q.Usernames[i])
}
type userRow struct {
User models.User `db:"hmn_user"`
AvatarAsset *models.Asset `db:"avatar"`
}
var qb db.QueryBuilder
qb.Add(`
SELECT $columns
FROM
hmn_user
LEFT JOIN asset AS avatar ON avatar.id = hmn_user.avatar_asset_id
WHERE
TRUE
`)
if len(q.UserIDs) > 0 {
qb.Add(`AND hmn_user.id = ANY($?)`, q.UserIDs)
}
if len(q.Usernames) > 0 {
qb.Add(`AND LOWER(hmn_user.username) = ANY($?)`, q.Usernames)
}
if !q.AnyStatus {
if currentUser == nil {
qb.Add(`AND hmn_user.status = $?`, models.UserStatusApproved)
} else if !currentUser.IsStaff {
qb.Add(
`
AND (
hmn_user.status = $? -- user is Approved
OR hmn_user.id = $? -- getting self
)
`,
models.UserStatusApproved,
currentUserID,
)
}
}
userRows, err := db.Query[userRow](ctx, dbConn, qb.String(), qb.Args()...)
if err != nil {
return nil, oops.New(err, "failed to fetch users")
}
result := make([]*models.User, len(userRows))
for i, row := range userRows {
user := row.User
user.AvatarAsset = row.AvatarAsset
result[i] = &user
}
return result, nil
}
/*
Fetches a single user and related data. A wrapper around FetchUsers.
As with FetchUsers, provide as much information as you know to get the
most correct results.
Returns db.NotFound if no result is found.
*/
func FetchUser(
ctx context.Context,
dbConn db.ConnOrTx,
currentUser *models.User,
userID int,
q UsersQuery,
) (*models.User, error) {
q.UserIDs = []int{userID}
res, err := FetchUsers(ctx, dbConn, currentUser, q)
if err != nil {
return nil, oops.New(err, "failed to fetch user")
}
if len(res) == 0 {
return nil, db.NotFound
}
return res[0], nil
}
/*
Fetches a single user and related data. A wrapper around FetchUsers.
As with FetchUsers, provide as much information as you know to get the
most correct results.
Returns db.NotFound if no result is found.
*/
func FetchUserByUsername(
ctx context.Context,
dbConn db.ConnOrTx,
currentUser *models.User,
username string,
q UsersQuery,
) (*models.User, error) {
q.Usernames = []string{username}
res, err := FetchUsers(ctx, dbConn, currentUser, q)
if err != nil {
return nil, oops.New(err, "failed to fetch user")
}
if len(res) == 0 {
return nil, db.NotFound
}
return res[0], nil
}
// NOTE(ben): Someday we can add CountUsers...I don't have a need for it right now.

101
src/hmns3/hmns3.go Normal file
View File

@ -0,0 +1,101 @@
package hmns3
import (
"context"
"errors"
"fmt"
"io"
"io/fs"
"net/http"
"os"
"path/filepath"
"strings"
"git.handmade.network/hmn/hmn/src/config"
"git.handmade.network/hmn/hmn/src/jobs"
"git.handmade.network/hmn/hmn/src/logging"
"git.handmade.network/hmn/hmn/src/utils"
"github.com/rs/zerolog"
)
const dir = "./tmp/s3"
type server struct {
log zerolog.Logger
}
func StartServer(ctx context.Context) jobs.Job {
if !config.Config.DigitalOcean.RunFakeServer {
return jobs.Noop()
}
utils.Must0(os.MkdirAll(dir, fs.ModePerm))
s := server{
log: logging.ExtractLogger(ctx).With().
Str("module", "S3 server").
Logger(),
}
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
s.getObject(w, r)
} else if r.Method == http.MethodPut {
s.putObject(w, r)
} else {
panic("Unimplemented method!")
}
})
job := jobs.New()
srv := http.Server{
Addr: config.Config.DigitalOcean.FakeAddr,
}
s.log.Info().Msg("Starting local S3 server")
go func() {
defer job.Done()
err := srv.ListenAndServe()
if err != nil {
if errors.Is(err, http.ErrServerClosed) {
// This is normal and fine
} else {
panic(err)
}
}
}()
go func() {
<-ctx.Done()
s.log.Info().Msg("Shutting down local S3 server")
srv.Shutdown(context.Background())
}()
return job
}
func (s *server) getObject(w http.ResponseWriter, r *http.Request) {
bucket, key := bucketKey(r)
file := utils.Must1(os.Open(filepath.Join(dir, bucket, key)))
io.Copy(w, file)
}
func (s *server) putObject(w http.ResponseWriter, r *http.Request) {
bucket, key := bucketKey(r)
w.Header().Set("Location", fmt.Sprintf("/%s", bucket))
utils.Must0(os.MkdirAll(filepath.Join(dir, bucket), fs.ModePerm))
if key != "" {
file := utils.Must1(os.Create(filepath.Join(dir, bucket, key)))
io.Copy(file, r.Body)
}
}
func bucketKey(r *http.Request) (string, string) {
slashIdx := strings.IndexByte(r.URL.Path[1:], '/')
if slashIdx == -1 {
return r.URL.Path[1:], ""
} else {
return r.URL.Path[1 : 1+slashIdx], strings.ReplaceAll(r.URL.Path[2+slashIdx:], "/", "~")
}
}

View File

@ -26,10 +26,18 @@ func TestUrl(t *testing.T) {
})
}
var hmn = HMNProjectContext
var hero = UrlContext{
PersonalProject: false,
ProjectID: 2,
ProjectSlug: "hero",
ProjectName: "Handmade Hero",
}
func TestHomepage(t *testing.T) {
AssertRegexMatch(t, BuildHomepage(), RegexHomepage, nil)
AssertRegexMatch(t, BuildProjectHomepage("hero"), RegexHomepage, nil)
AssertSubdomain(t, BuildProjectHomepage("hero"), "hero")
AssertRegexMatch(t, hero.BuildHomepage(), RegexHomepage, nil)
AssertSubdomain(t, hero.BuildHomepage(), "hero")
}
func TestShowcase(t *testing.T) {
@ -87,7 +95,6 @@ func TestPasswordReset(t *testing.T) {
func TestStaticPages(t *testing.T) {
AssertRegexMatch(t, BuildManifesto(), RegexManifesto, nil)
AssertRegexMatch(t, BuildAbout(), RegexAbout, nil)
AssertRegexMatch(t, BuildCodeOfConduct(), RegexCodeOfConduct, nil)
AssertRegexMatch(t, BuildCommunicationGuidelines(), RegexCommunicationGuidelines, nil)
AssertRegexMatch(t, BuildContactPage(), RegexContactPage, nil)
AssertRegexMatch(t, BuildMonthlyUpdatePolicy(), RegexMonthlyUpdatePolicy, nil)
@ -133,12 +140,12 @@ func TestProjectNew(t *testing.T) {
AssertRegexMatch(t, BuildProjectNew(), RegexProjectNew, nil)
}
func TestProjectNotApproved(t *testing.T) {
AssertRegexMatch(t, BuildProjectNotApproved("test"), RegexProjectNotApproved, map[string]string{"slug": "test"})
func TestPersonalProject(t *testing.T) {
AssertRegexMatch(t, BuildPersonalProject(123, "test"), RegexPersonalProject, nil)
}
func TestProjectEdit(t *testing.T) {
AssertRegexMatch(t, BuildProjectEdit("test", "foo"), RegexProjectEdit, map[string]string{"slug": "test"})
AssertRegexMatch(t, hero.BuildProjectEdit("foo"), RegexProjectEdit, nil)
}
func TestPodcast(t *testing.T) {
@ -166,101 +173,100 @@ func TestPodcastRSS(t *testing.T) {
}
func TestForum(t *testing.T) {
AssertRegexMatch(t, BuildForum("", nil, 1), RegexForum, nil)
AssertRegexMatch(t, BuildForum("", []string{"wip"}, 2), RegexForum, map[string]string{"subforums": "wip", "page": "2"})
AssertRegexMatch(t, BuildForum("", []string{"sub", "wip"}, 2), RegexForum, map[string]string{"subforums": "sub/wip", "page": "2"})
AssertSubdomain(t, BuildForum("hmn", nil, 1), "")
AssertSubdomain(t, BuildForum("", nil, 1), "")
AssertSubdomain(t, BuildForum("hero", nil, 1), "hero")
assert.Panics(t, func() { BuildForum("", nil, 0) })
assert.Panics(t, func() { BuildForum("", []string{"", "wip"}, 1) })
assert.Panics(t, func() { BuildForum("", []string{" ", "wip"}, 1) })
assert.Panics(t, func() { BuildForum("", []string{"wip/jobs"}, 1) })
AssertRegexMatch(t, hmn.BuildForum(nil, 1), RegexForum, nil)
AssertRegexMatch(t, hmn.BuildForum([]string{"wip"}, 2), RegexForum, map[string]string{"subforums": "wip", "page": "2"})
AssertRegexMatch(t, hmn.BuildForum([]string{"sub", "wip"}, 2), RegexForum, map[string]string{"subforums": "sub/wip", "page": "2"})
AssertSubdomain(t, hmn.BuildForum(nil, 1), "")
AssertSubdomain(t, hero.BuildForum(nil, 1), "hero")
assert.Panics(t, func() { hmn.BuildForum(nil, 0) })
assert.Panics(t, func() { hmn.BuildForum([]string{"", "wip"}, 1) })
assert.Panics(t, func() { hmn.BuildForum([]string{" ", "wip"}, 1) })
assert.Panics(t, func() { hmn.BuildForum([]string{"wip/jobs"}, 1) })
}
func TestForumNewThread(t *testing.T) {
AssertRegexMatch(t, BuildForumNewThread("", []string{"sub", "wip"}, false), RegexForumNewThread, map[string]string{"subforums": "sub/wip"})
AssertRegexMatch(t, BuildForumNewThread("", []string{"sub", "wip"}, true), RegexForumNewThreadSubmit, map[string]string{"subforums": "sub/wip"})
AssertRegexMatch(t, hmn.BuildForumNewThread([]string{"sub", "wip"}, false), RegexForumNewThread, map[string]string{"subforums": "sub/wip"})
AssertRegexMatch(t, hmn.BuildForumNewThread([]string{"sub", "wip"}, true), RegexForumNewThreadSubmit, map[string]string{"subforums": "sub/wip"})
}
func TestForumThread(t *testing.T) {
AssertRegexMatch(t, BuildForumThread("", nil, 1, "", 1), RegexForumThread, map[string]string{"threadid": "1"})
AssertRegexMatch(t, BuildForumThread("", nil, 1, "thread/title/123http://", 2), RegexForumThread, map[string]string{"threadid": "1", "page": "2"})
AssertRegexMatch(t, BuildForumThreadWithPostHash("", nil, 1, "thread/title/123http://", 2, 123), RegexForumThread, map[string]string{"threadid": "1", "page": "2"})
AssertSubdomain(t, BuildForumThread("hero", nil, 1, "", 1), "hero")
assert.Panics(t, func() { BuildForumThread("", nil, -1, "", 1) })
assert.Panics(t, func() { BuildForumThread("", nil, 1, "", -1) })
AssertRegexMatch(t, hmn.BuildForumThread(nil, 1, "", 1), RegexForumThread, map[string]string{"threadid": "1"})
AssertRegexMatch(t, hmn.BuildForumThread(nil, 1, "thread/title/123http://", 2), RegexForumThread, map[string]string{"threadid": "1", "page": "2"})
AssertRegexMatch(t, hmn.BuildForumThreadWithPostHash(nil, 1, "thread/title/123http://", 2, 123), RegexForumThread, map[string]string{"threadid": "1", "page": "2"})
AssertSubdomain(t, hero.BuildForumThread(nil, 1, "", 1), "hero")
assert.Panics(t, func() { hmn.BuildForumThread(nil, -1, "", 1) })
assert.Panics(t, func() { hmn.BuildForumThread(nil, 1, "", -1) })
}
func TestForumPost(t *testing.T) {
AssertRegexMatch(t, BuildForumPost("", nil, 1, 2), RegexForumPost, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, BuildForumPost("", nil, 1, 2), RegexForumThread)
AssertSubdomain(t, BuildForumPost("hero", nil, 1, 2), "hero")
assert.Panics(t, func() { BuildForumPost("", nil, 1, -1) })
AssertRegexMatch(t, hmn.BuildForumPost(nil, 1, 2), RegexForumPost, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, hmn.BuildForumPost(nil, 1, 2), RegexForumThread)
AssertSubdomain(t, hero.BuildForumPost(nil, 1, 2), "hero")
assert.Panics(t, func() { hmn.BuildForumPost(nil, 1, -1) })
}
func TestForumPostDelete(t *testing.T) {
AssertRegexMatch(t, BuildForumPostDelete("", nil, 1, 2), RegexForumPostDelete, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, BuildForumPostDelete("", nil, 1, 2), RegexForumPost)
AssertSubdomain(t, BuildForumPostDelete("hero", nil, 1, 2), "hero")
AssertRegexMatch(t, hmn.BuildForumPostDelete(nil, 1, 2), RegexForumPostDelete, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, hmn.BuildForumPostDelete(nil, 1, 2), RegexForumPost)
AssertSubdomain(t, hero.BuildForumPostDelete(nil, 1, 2), "hero")
}
func TestForumPostEdit(t *testing.T) {
AssertRegexMatch(t, BuildForumPostEdit("", nil, 1, 2), RegexForumPostEdit, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, BuildForumPostEdit("", nil, 1, 2), RegexForumPost)
AssertSubdomain(t, BuildForumPostEdit("hero", nil, 1, 2), "hero")
AssertRegexMatch(t, hmn.BuildForumPostEdit(nil, 1, 2), RegexForumPostEdit, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, hmn.BuildForumPostEdit(nil, 1, 2), RegexForumPost)
AssertSubdomain(t, hero.BuildForumPostEdit(nil, 1, 2), "hero")
}
func TestForumPostReply(t *testing.T) {
AssertRegexMatch(t, BuildForumPostReply("", nil, 1, 2), RegexForumPostReply, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, BuildForumPostReply("", nil, 1, 2), RegexForumPost)
AssertSubdomain(t, BuildForumPostReply("hero", nil, 1, 2), "hero")
AssertRegexMatch(t, hmn.BuildForumPostReply(nil, 1, 2), RegexForumPostReply, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, hmn.BuildForumPostReply(nil, 1, 2), RegexForumPost)
AssertSubdomain(t, hero.BuildForumPostReply(nil, 1, 2), "hero")
}
func TestBlog(t *testing.T) {
AssertRegexMatch(t, BuildBlog("", 1), RegexBlog, nil)
AssertRegexMatch(t, BuildBlog("", 2), RegexBlog, map[string]string{"page": "2"})
AssertSubdomain(t, BuildBlog("hero", 1), "hero")
AssertRegexMatch(t, hmn.BuildBlog(1), RegexBlog, nil)
AssertRegexMatch(t, hmn.BuildBlog(2), RegexBlog, map[string]string{"page": "2"})
AssertSubdomain(t, hero.BuildBlog(1), "hero")
}
func TestBlogNewThread(t *testing.T) {
AssertRegexMatch(t, BuildBlogNewThread(""), RegexBlogNewThread, nil)
AssertSubdomain(t, BuildBlogNewThread(""), "")
AssertRegexMatch(t, BuildBlogNewThread("hero"), RegexBlogNewThread, nil)
AssertSubdomain(t, BuildBlogNewThread("hero"), "hero")
AssertRegexMatch(t, hmn.BuildBlogNewThread(), RegexBlogNewThread, nil)
AssertSubdomain(t, hmn.BuildBlogNewThread(), "")
AssertRegexMatch(t, hero.BuildBlogNewThread(), RegexBlogNewThread, nil)
AssertSubdomain(t, hero.BuildBlogNewThread(), "hero")
}
func TestBlogThread(t *testing.T) {
AssertRegexMatch(t, BuildBlogThread("", 1, ""), RegexBlogThread, map[string]string{"threadid": "1"})
AssertRegexMatch(t, BuildBlogThread("", 1, ""), RegexBlogThread, map[string]string{"threadid": "1"})
AssertRegexMatch(t, BuildBlogThread("", 1, "title/bla/http://"), RegexBlogThread, map[string]string{"threadid": "1"})
AssertRegexMatch(t, BuildBlogThreadWithPostHash("", 1, "title/bla/http://", 123), RegexBlogThread, map[string]string{"threadid": "1"})
AssertRegexNoMatch(t, BuildBlogThread("", 1, ""), RegexBlog)
AssertSubdomain(t, BuildBlogThread("hero", 1, ""), "hero")
AssertRegexMatch(t, hmn.BuildBlogThread(1, ""), RegexBlogThread, map[string]string{"threadid": "1"})
AssertRegexMatch(t, hmn.BuildBlogThread(1, ""), RegexBlogThread, map[string]string{"threadid": "1"})
AssertRegexMatch(t, hmn.BuildBlogThread(1, "title/bla/http://"), RegexBlogThread, map[string]string{"threadid": "1"})
AssertRegexMatch(t, hmn.BuildBlogThreadWithPostHash(1, "title/bla/http://", 123), RegexBlogThread, map[string]string{"threadid": "1"})
AssertRegexNoMatch(t, hmn.BuildBlogThread(1, ""), RegexBlog)
AssertSubdomain(t, hero.BuildBlogThread(1, ""), "hero")
}
func TestBlogPost(t *testing.T) {
AssertRegexMatch(t, BuildBlogPost("", 1, 2), RegexBlogPost, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, BuildBlogPost("", 1, 2), RegexBlogThread)
AssertSubdomain(t, BuildBlogPost("hero", 1, 2), "hero")
AssertRegexMatch(t, hmn.BuildBlogPost(1, 2), RegexBlogPost, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, hmn.BuildBlogPost(1, 2), RegexBlogThread)
AssertSubdomain(t, hero.BuildBlogPost(1, 2), "hero")
}
func TestBlogPostDelete(t *testing.T) {
AssertRegexMatch(t, BuildBlogPostDelete("", 1, 2), RegexBlogPostDelete, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, BuildBlogPostDelete("", 1, 2), RegexBlogPost)
AssertSubdomain(t, BuildBlogPostDelete("hero", 1, 2), "hero")
AssertRegexMatch(t, hmn.BuildBlogPostDelete(1, 2), RegexBlogPostDelete, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, hmn.BuildBlogPostDelete(1, 2), RegexBlogPost)
AssertSubdomain(t, hero.BuildBlogPostDelete(1, 2), "hero")
}
func TestBlogPostEdit(t *testing.T) {
AssertRegexMatch(t, BuildBlogPostEdit("", 1, 2), RegexBlogPostEdit, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, BuildBlogPostEdit("", 1, 2), RegexBlogPost)
AssertSubdomain(t, BuildBlogPostEdit("hero", 1, 2), "hero")
AssertRegexMatch(t, hmn.BuildBlogPostEdit(1, 2), RegexBlogPostEdit, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, hmn.BuildBlogPostEdit(1, 2), RegexBlogPost)
AssertSubdomain(t, hero.BuildBlogPostEdit(1, 2), "hero")
}
func TestBlogPostReply(t *testing.T) {
AssertRegexMatch(t, BuildBlogPostReply("", 1, 2), RegexBlogPostReply, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, BuildBlogPostReply("", 1, 2), RegexBlogPost)
AssertSubdomain(t, BuildBlogPostReply("hero", 1, 2), "hero")
AssertRegexMatch(t, hmn.BuildBlogPostReply(1, 2), RegexBlogPostReply, map[string]string{"threadid": "1", "postid": "2"})
AssertRegexNoMatch(t, hmn.BuildBlogPostReply(1, 2), RegexBlogPost)
AssertSubdomain(t, hero.BuildBlogPostReply(1, 2), "hero")
}
func TestLibrary(t *testing.T) {
@ -280,20 +286,20 @@ func TestLibraryResource(t *testing.T) {
}
func TestEpisodeGuide(t *testing.T) {
AssertRegexMatch(t, BuildEpisodeList("hero", ""), RegexEpisodeList, map[string]string{"topic": ""})
AssertRegexMatch(t, BuildEpisodeList("hero", "code"), RegexEpisodeList, map[string]string{"topic": "code"})
AssertSubdomain(t, BuildEpisodeList("hero", "code"), "hero")
AssertRegexMatch(t, hero.BuildEpisodeList(""), RegexEpisodeList, map[string]string{"topic": ""})
AssertRegexMatch(t, hero.BuildEpisodeList("code"), RegexEpisodeList, map[string]string{"topic": "code"})
AssertSubdomain(t, hero.BuildEpisodeList("code"), "hero")
AssertRegexMatch(t, BuildEpisode("hero", "code", "day001"), RegexEpisode, map[string]string{"topic": "code", "episode": "day001"})
AssertSubdomain(t, BuildEpisode("hero", "code", "day001"), "hero")
AssertRegexMatch(t, hero.BuildEpisode("code", "day001"), RegexEpisode, map[string]string{"topic": "code", "episode": "day001"})
AssertSubdomain(t, hero.BuildEpisode("code", "day001"), "hero")
AssertRegexMatch(t, BuildCineraIndex("hero", "code"), RegexCineraIndex, map[string]string{"topic": "code"})
AssertSubdomain(t, BuildCineraIndex("hero", "code"), "hero")
AssertRegexMatch(t, hero.BuildCineraIndex("code"), RegexCineraIndex, map[string]string{"topic": "code"})
AssertSubdomain(t, hero.BuildCineraIndex("code"), "hero")
}
func TestAssetUpload(t *testing.T) {
AssertRegexMatch(t, BuildAssetUpload("hero"), RegexAssetUpload, nil)
AssertSubdomain(t, BuildAssetUpload("hero"), "hero")
AssertRegexMatch(t, hero.BuildAssetUpload(), RegexAssetUpload, nil)
AssertSubdomain(t, hero.BuildAssetUpload(), "hero")
}
func TestProjectCSS(t *testing.T) {
@ -308,6 +314,10 @@ func TestAPICheckUsername(t *testing.T) {
AssertRegexMatch(t, BuildAPICheckUsername(), RegexAPICheckUsername, nil)
}
func TestTwitchEventSubCallback(t *testing.T) {
AssertRegexMatch(t, BuildTwitchEventSubCallback(), RegexTwitchEventSubCallback, nil)
}
func TestPublic(t *testing.T) {
AssertRegexMatch(t, BuildPublic("test", false), RegexPublic, nil)
AssertRegexMatch(t, BuildPublic("/test", true), RegexPublic, nil)
@ -324,8 +334,8 @@ func TestPublic(t *testing.T) {
}
func TestForumMarkRead(t *testing.T) {
AssertRegexMatch(t, BuildForumMarkRead("hero", 5), RegexForumMarkRead, map[string]string{"sfid": "5"})
AssertSubdomain(t, BuildForumMarkRead("hero", 5), "hero")
AssertRegexMatch(t, hero.BuildForumMarkRead(5), RegexForumMarkRead, map[string]string{"sfid": "5"})
AssertSubdomain(t, hero.BuildForumMarkRead(5), "hero")
}
func TestS3Asset(t *testing.T) {

40
src/jobs/jobs.go Normal file
View File

@ -0,0 +1,40 @@
package jobs
type Job struct {
C <-chan struct{}
rawC chan struct{}
}
func New() Job {
return newFromChannel(make(chan struct{}))
}
func (j *Job) Done() {
close(j.rawC)
}
// Combines multiple jobs into one.
func Zip(jobs ...Job) Job {
out := make(chan struct{})
go func() {
for _, job := range jobs {
<-job.C
}
close(out)
}()
return newFromChannel(out)
}
// Returns a job that is already done.
func Noop() Job {
job := New()
job.Done()
return job
}
func newFromChannel(c chan struct{}) Job {
return Job{
C: c,
rawC: c,
}
}

View File

@ -5,7 +5,7 @@ import (
_ "embed"
"fmt"
"os"
"os/exec"
"os/user"
"path/filepath"
"sort"
"strings"
@ -24,6 +24,11 @@ import (
var listMigrations bool
func init() {
dbCommand := &cobra.Command{
Use: "db",
Short: "Database-related commands",
}
migrateCommand := &cobra.Command{
Use: "migrate [target migration id]",
Short: "Run database migrations",
@ -64,6 +69,15 @@ func init() {
},
}
seedCommand := &cobra.Command{
Use: "seed",
Short: "Resets the db and populates it with sample data.",
Run: func(cmd *cobra.Command, args []string) {
ResetDB()
SampleSeed()
},
}
seedFromFileCommand := &cobra.Command{
Use: "seedfile <filename>",
Short: "Resets the db and runs the seed file.",
@ -74,13 +88,16 @@ func init() {
os.Exit(1)
}
ResetDB()
SeedFromFile(args[0])
},
}
website.WebsiteCommand.AddCommand(migrateCommand)
website.WebsiteCommand.AddCommand(makeMigrationCommand)
website.WebsiteCommand.AddCommand(seedFromFileCommand)
website.WebsiteCommand.AddCommand(dbCommand)
dbCommand.AddCommand(migrateCommand)
dbCommand.AddCommand(makeMigrationCommand)
dbCommand.AddCommand(seedCommand)
dbCommand.AddCommand(seedFromFileCommand)
}
func getSortedMigrationVersions() []types.MigrationVersion {
@ -134,10 +151,19 @@ func ListMigrations() {
}
}
func LatestVersion() types.MigrationVersion {
allVersions := getSortedMigrationVersions()
return allVersions[len(allVersions)-1]
}
// Migrates either forward or backward to the selected migration version. You probably want to
// use LatestVersion to get the most recent migration.
func Migrate(targetVersion types.MigrationVersion) {
ctx := context.Background() // In the future, this could actually do something cool.
conn := db.NewConn()
conn := db.NewConnWithConfig(config.PostgresConfig{
LogLevel: pgx.LogLevelWarn,
})
defer conn.Close(ctx)
// create migration table
@ -177,7 +203,7 @@ func Migrate(targetVersion types.MigrationVersion) {
allVersions := getSortedMigrationVersions()
if targetVersion.IsZero() {
targetVersion = allVersions[len(allVersions)-1]
targetVersion = LatestVersion()
}
currentIndex := -1
@ -290,84 +316,135 @@ func MakeMigration(name, description string) {
fmt.Println(path)
}
// Applies a cloned db to the local db.
// Applies the seed after the migration specified in `afterMigration`.
// NOTE(asaf): The db role specified in the config must have the CREATEDB attribute! `ALTER ROLE hmn WITH CREATEDB;`
func SeedFromFile(seedFile string) {
file, err := os.Open(seedFile)
if err != nil {
panic(fmt.Errorf("couldn't open seed file %s: %w", seedFile, err))
}
file.Close()
func ResetDB() {
fmt.Println("Resetting database...")
{
ctx := context.Background()
// NOTE(asaf): We connect to db "template1", because we have to connect to something other than our own db in order to drop it.
template1DSN := fmt.Sprintf("user=%s password=%s host=%s port=%d dbname=%s",
config.Config.Postgres.User,
config.Config.Postgres.Password,
config.Config.Postgres.Hostname,
config.Config.Postgres.Port,
"template1", // NOTE(asaf): template1 must always exist in postgres, as it's the db that gets cloned when you create new DBs
)
// NOTE(asaf): We have to use the low-level API of pgconn, because the pgx Exec always wraps the query in a transaction.
lowLevelConn, err := pgconn.Connect(ctx, template1DSN)
if err != nil {
panic(fmt.Errorf("failed to connect to db: %w", err))
}
defer lowLevelConn.Close(ctx)
result := lowLevelConn.ExecParams(ctx, fmt.Sprintf("DROP DATABASE %s", config.Config.Postgres.DbName), nil, nil, nil, nil)
_, err = result.Close()
ctx := context.Background()
// Create the HMN database user
{
type pgCredentials struct {
User string
Password string
}
credentials := []pgCredentials{
{config.Config.Postgres.User, config.Config.Postgres.Password}, // Existing HMN user
{getSystemUsername(), ""}, // Postgres.app on Mac
}
var workingCred pgCredentials
var createUserConn *pgconn.PgConn
var connErrors []error
for _, cred := range credentials {
// NOTE(asaf): We have to use the low-level API of pgconn, because the pgx Exec always wraps the query in a transaction.
var err error
createUserConn, err = connectLowLevel(ctx, cred.User, cred.Password)
if err == nil {
workingCred = cred
break
} else {
connErrors = append(connErrors, err)
}
}
if createUserConn == nil {
fmt.Println("Failed to connect to the db to reset it.")
fmt.Println("The following errors occurred for each attempted set of credentials:")
for _, err := range connErrors {
fmt.Printf("- %v\n", err)
}
fmt.Println()
fmt.Println("If this is a local development environment, please let us know what platform you")
fmt.Println("are using and how you installed Postgres. We want to try and streamline the setup")
fmt.Println("process for you.")
fmt.Println()
fmt.Println("If on the other hand this is a real deployment, please go into psql and manually")
fmt.Println("create the user:")
fmt.Println()
fmt.Println(" CREATE USER <username> WITH")
fmt.Println(" ENCRYPTED PASSWORD '<password>'")
fmt.Println(" CREATEDB;")
fmt.Println()
fmt.Println("and add the username and password to your config.")
os.Exit(1)
}
defer createUserConn.Close(ctx)
// Create the HMN user
{
userExists := workingCred.User == config.Config.Postgres.User && workingCred.Password == config.Config.Postgres.Password
if !userExists {
result := createUserConn.ExecParams(ctx, fmt.Sprintf(`
CREATE USER %s WITH
ENCRYPTED PASSWORD '%s'
CREATEDB
`, config.Config.Postgres.User, config.Config.Postgres.Password), nil, nil, nil, nil)
_, err := result.Close()
if err != nil {
panic(fmt.Errorf("failed to create HMN user: %w", err))
}
}
}
}
// Connect as the HMN user
conn, err := connectLowLevel(ctx, config.Config.Postgres.User, config.Config.Postgres.Password)
if err != nil {
panic(fmt.Errorf("failed to connect to db: %w", err))
}
// Disconnect all other users
{
result := conn.ExecParams(ctx, fmt.Sprintf(`
SELECT pg_terminate_backend(pid)
FROM pg_stat_activity
WHERE datname IN ('%s', 'template1') AND pid <> pg_backend_pid()
`, config.Config.Postgres.DbName), nil, nil, nil, nil)
_, err := result.Close()
if err != nil {
panic(fmt.Errorf("failed to disconnect other users: %w", err))
}
}
// Drop the database
{
result := conn.ExecParams(ctx, fmt.Sprintf("DROP DATABASE %s", config.Config.Postgres.DbName), nil, nil, nil, nil)
_, err := result.Close()
pgErr, isPgError := err.(*pgconn.PgError)
if err != nil {
if !(isPgError && pgErr.SQLState() == "3D000") { // NOTE(asaf): 3D000 means "Database does not exist"
panic(fmt.Errorf("failed to drop db: %w", err))
}
}
}
result = lowLevelConn.ExecParams(ctx, fmt.Sprintf("CREATE DATABASE %s", config.Config.Postgres.DbName), nil, nil, nil, nil)
_, err = result.Close()
// Create the database again
{
result := conn.ExecParams(ctx, fmt.Sprintf("CREATE DATABASE %s", config.Config.Postgres.DbName), nil, nil, nil, nil)
_, err := result.Close()
if err != nil {
panic(fmt.Errorf("failed to create db: %w", err))
}
}
fmt.Println("Executing seed...")
cmd := exec.Command("pg_restore",
"--single-transaction",
"--dbname", config.Config.Postgres.DSN(),
seedFile,
fmt.Println("Database reset successfully.")
}
func connectLowLevel(ctx context.Context, username, password string) (*pgconn.PgConn, error) {
// NOTE(asaf): We connect to db "template1", because we have to connect to something other than our own db in order to drop it.
template1DSN := fmt.Sprintf("user=%s password=%s host=%s port=%d dbname=%s",
username,
password,
config.Config.Postgres.Hostname,
config.Config.Postgres.Port,
"template1", // NOTE(asaf): template1 must always exist in postgres, as it's the db that gets cloned when you create new DBs
)
fmt.Println("Running command:", cmd)
if output, err := cmd.CombinedOutput(); err != nil {
fmt.Print(string(output))
panic(fmt.Errorf("failed to execute seed: %w", err))
return pgconn.Connect(ctx, template1DSN)
}
func getSystemUsername() string {
u, err := user.Current()
if err != nil {
return ""
}
fmt.Println("Done! You may want to migrate forward from here.")
ListMigrations()
}
// NOTE(asaf): This will be useful for open-sourcing the website, but is not yet necessary.
// Creates only what's necessary for a fresh deployment with no data
// TODO(opensource)
func BareMinimumSeed() {
}
// NOTE(asaf): This will be useful for open-sourcing the website, but is not yet necessary.
// Creates enough data for development
// TODO(opensource)
func SampleSeed() {
// admin := CreateAdminUser("admin", "12345678")
// user := CreateUser("regular_user", "12345678")
// hmnProject := CreateProject("hmn", "Handmade Network")
// Create category
// Create thread
// Create accepted user project
// Create pending user project
// Create showcase items
// Create codelanguages
// Create library and library resources
return u.Username
}

View File

@ -110,7 +110,7 @@ func (m PersonalProjects) Up(ctx context.Context, tx pgx.Tx) error {
// Port "jam snippets" to use a tag
//
jamTagId, err := db.QueryInt(ctx, tx, `INSERT INTO tags (text) VALUES ('wheeljam') RETURNING id`)
jamTagId, err := db.QueryOneScalar[int](ctx, tx, `INSERT INTO tags (text) VALUES ('wheeljam') RETURNING id`)
if err != nil {
return oops.New(err, "failed to create jam tag")
}

View File

@ -0,0 +1,146 @@
package migrations
import (
"context"
"time"
"git.handmade.network/hmn/hmn/src/migration/types"
"git.handmade.network/hmn/hmn/src/oops"
"github.com/jackc/pgx/v4"
)
func init() {
registerMigration(RenameEverything{})
}
type RenameEverything struct{}
func (m RenameEverything) Version() types.MigrationVersion {
return types.MigrationVersion(time.Date(2022, 5, 6, 22, 17, 26, 0, time.UTC))
}
func (m RenameEverything) Name() string {
return "RenameEverything"
}
func (m RenameEverything) Description() string {
return "Rename all the tables, and remove the ones we no longer need"
}
func (m RenameEverything) Up(ctx context.Context, tx pgx.Tx) error {
// Drop unused tables
_, err := tx.Exec(ctx, `
DROP TABLE
auth_permission,
django_content_type,
django_migrations,
django_site,
handmade_blacklistemail,
handmade_blacklisthostname,
handmade_codelanguage,
handmade_communicationchoice,
handmade_communicationchoicelist,
handmade_communicationsubcategory,
handmade_communicationsubthread,
handmade_kunenathread,
handmade_license,
handmade_license_texts,
handmade_project_languages,
handmade_project_licenses
`)
if err != nil {
return oops.New(err, "failed to drop unused tables")
}
// Rename everything!!
_, err = tx.Exec(ctx, `
ALTER TABLE auth_user RENAME TO hmn_user;
ALTER TABLE discord_outgoingmessages RENAME TO discord_outgoing_message;
ALTER TABLE handmade_asset RENAME TO asset;
ALTER TABLE handmade_discordmessage RENAME TO discord_message;
ALTER TABLE handmade_discordmessageattachment RENAME TO discord_message_attachment;
ALTER TABLE handmade_discordmessagecontent RENAME TO discord_message_content;
ALTER TABLE handmade_discordmessageembed RENAME TO discord_message_embed;
ALTER TABLE handmade_discorduser RENAME TO discord_user;
ALTER TABLE handmade_imagefile RENAME TO image_file;
ALTER TABLE handmade_librarymediatype RENAME TO library_media_type;
ALTER TABLE handmade_libraryresource RENAME TO library_resource;
ALTER TABLE handmade_libraryresource_media_types RENAME TO library_resource_media_type;
ALTER TABLE handmade_libraryresource_topics RENAME TO library_resource_topic;
ALTER TABLE handmade_libraryresourcestar RENAME TO library_resource_star;
ALTER TABLE handmade_librarytopic RENAME TO library_topic;
ALTER TABLE handmade_links RENAME TO link;
ALTER TABLE handmade_onetimetoken RENAME TO one_time_token;
ALTER TABLE handmade_otherfile RENAME TO other_file;
ALTER TABLE handmade_podcast RENAME TO podcast;
ALTER TABLE handmade_podcastepisode RENAME TO podcast_episode;
ALTER TABLE handmade_post RENAME TO post;
ALTER TABLE handmade_post_asset_usage RENAME TO post_asset_usage;
ALTER TABLE handmade_postversion RENAME TO post_version;
ALTER TABLE handmade_project RENAME TO project;
ALTER TABLE handmade_project_downloads RENAME TO project_download;
ALTER TABLE handmade_project_screenshots RENAME TO project_screenshot;
ALTER TABLE handmade_snippet RENAME TO snippet;
ALTER TABLE handmade_subforum RENAME TO subforum;
ALTER TABLE handmade_subforumlastreadinfo RENAME TO subforum_last_read_info;
ALTER TABLE handmade_thread RENAME TO thread;
ALTER TABLE handmade_threadlastreadinfo RENAME TO thread_last_read_info;
ALTER TABLE handmade_user_projects RENAME TO user_project;
ALTER TABLE sessions RENAME TO session;
ALTER TABLE snippet_tags RENAME TO snippet_tag;
ALTER TABLE tags RENAME TO tag;
ALTER TABLE twitch_streams RENAME TO twitch_stream;
ALTER SEQUENCE auth_user_id_seq RENAME TO hmn_user_id_seq;
ALTER SEQUENCE discord_outgoingmessages_id_seq RENAME TO discord_outgoing_message_id_seq;
ALTER SEQUENCE handmade_category_id_seq RENAME TO subforum_id_seq;
ALTER SEQUENCE handmade_categorylastreadinfo_id_seq RENAME TO subforum_last_read_info_id_seq;
ALTER SEQUENCE handmade_discord_id_seq RENAME TO discord_user_id_seq;
ALTER SEQUENCE handmade_discordmessageembed_id_seq RENAME TO discord_message_embed_id_seq;
ALTER SEQUENCE handmade_imagefile_id_seq RENAME TO image_file_id_seq;
ALTER SEQUENCE handmade_librarymediatype_id_seq RENAME TO library_media_type_id_seq;
ALTER SEQUENCE handmade_libraryresource_id_seq RENAME TO library_resource_id_seq;
ALTER SEQUENCE handmade_libraryresource_media_types_id_seq RENAME TO library_resource_media_type_id_seq;
ALTER SEQUENCE handmade_libraryresource_topics_id_seq RENAME TO library_resource_topic_id_seq;
ALTER SEQUENCE handmade_libraryresourcestar_id_seq RENAME TO library_resource_star_id_seq;
ALTER SEQUENCE handmade_librarytopic_id_seq RENAME TO library_topic_id_seq;
ALTER SEQUENCE handmade_links_id_seq RENAME TO link_id_seq;
ALTER SEQUENCE handmade_onetimetoken_id_seq RENAME TO one_time_token_id_seq;
ALTER SEQUENCE handmade_otherfile_id_seq RENAME TO other_file_id_seq;
ALTER SEQUENCE handmade_podcast_id_seq RENAME TO podcast_id_seq;
ALTER SEQUENCE handmade_post_id_seq RENAME TO post_id_seq;
ALTER SEQUENCE handmade_postversion_id_seq RENAME TO post_version_id_seq;
ALTER SEQUENCE handmade_project_downloads_id_seq RENAME TO project_download_id_seq;
ALTER SEQUENCE handmade_project_id_seq RENAME TO project_id_seq;
ALTER SEQUENCE handmade_project_screenshots_id_seq RENAME TO project_screenshot_id_seq;
ALTER SEQUENCE handmade_snippet_id_seq RENAME TO snippet_id_seq;
ALTER SEQUENCE handmade_thread_id_seq RENAME TO thread_id_seq;
ALTER SEQUENCE handmade_threadlastreadinfo_id_seq RENAME TO thread_last_read_info_id_seq;
ALTER SEQUENCE tags_id_seq RENAME TO tag_id_seq;
CREATE OR REPLACE FUNCTION thread_type_for_post(int) RETURNS int AS $$
SELECT thread.type
FROM
public.post
JOIN public.thread ON post.thread_id = thread.id
WHERE post.id = $1
$$ LANGUAGE SQL;
CREATE OR REPLACE FUNCTION project_id_for_post(int) RETURNS int AS $$
SELECT thread.project_id
FROM
public.post
JOIN public.thread ON post.thread_id = thread.id
WHERE post.id = $1
$$ LANGUAGE SQL;
`)
if err != nil {
return oops.New(err, "failed to rename tables")
}
return nil
}
func (m RenameEverything) Down(ctx context.Context, tx pgx.Tx) error {
panic("Implement me")
}

View File

@ -0,0 +1,57 @@
package migrations
import (
"context"
"time"
"git.handmade.network/hmn/hmn/src/migration/types"
"git.handmade.network/hmn/hmn/src/oops"
"github.com/jackc/pgx/v4"
)
func init() {
registerMigration(AddPersistentVars{})
}
type AddPersistentVars struct{}
func (m AddPersistentVars) Version() types.MigrationVersion {
return types.MigrationVersion(time.Date(2022, 5, 26, 14, 45, 17, 0, time.UTC))
}
func (m AddPersistentVars) Name() string {
return "AddPersistentVars"
}
func (m AddPersistentVars) Description() string {
return "Create table for persistent_vars"
}
func (m AddPersistentVars) Up(ctx context.Context, tx pgx.Tx) error {
_, err := tx.Exec(ctx,
`
CREATE TABLE persistent_var (
name VARCHAR(255) NOT NULL,
value TEXT NOT NULL
);
CREATE UNIQUE INDEX persistent_var_name ON persistent_var (name);
`,
)
if err != nil {
return oops.New(err, "failed to create persistent_var table")
}
return nil
}
func (m AddPersistentVars) Down(ctx context.Context, tx pgx.Tx) error {
_, err := tx.Exec(ctx,
`
DROP INDEX persistent_var_name;
DROP TABLE persistent_var;
`,
)
if err != nil {
return oops.New(err, "failed to drop persistent_var table")
}
return nil
}

395
src/migration/seed.go Normal file
View File

@ -0,0 +1,395 @@
package migration
import (
"context"
"fmt"
"math/rand"
"os"
"os/exec"
"strings"
"time"
"git.handmade.network/hmn/hmn/src/auth"
"git.handmade.network/hmn/hmn/src/config"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/hmndata"
"git.handmade.network/hmn/hmn/src/models"
"git.handmade.network/hmn/hmn/src/oops"
"git.handmade.network/hmn/hmn/src/parsing"
"git.handmade.network/hmn/hmn/src/utils"
lorem "github.com/HandmadeNetwork/golorem"
"github.com/jackc/pgx/v4"
)
// Applies a cloned db to the local db.
// Applies the seed after the migration specified in `afterMigration`.
// NOTE(asaf): The db role specified in the config must have the CREATEDB attribute! `ALTER ROLE hmn WITH CREATEDB;`
func SeedFromFile(seedFile string) {
file, err := os.Open(seedFile)
if err != nil {
panic(fmt.Errorf("couldn't open seed file %s: %w", seedFile, err))
}
file.Close()
fmt.Println("Executing seed...")
cmd := exec.Command("pg_restore",
"--single-transaction",
"--dbname", config.Config.Postgres.DSN(),
seedFile,
)
fmt.Println("Running command:", cmd)
if output, err := cmd.CombinedOutput(); err != nil {
fmt.Print(string(output))
panic(fmt.Errorf("failed to execute seed: %w", err))
}
fmt.Println("Done! You may want to migrate forward from here.")
ListMigrations()
}
// Creates only what's necessary to get the site running. Not really very useful for
// local dev on its own; sample data makes things a lot better.
func BareMinimumSeed() *models.Project {
Migrate(LatestVersion())
ctx := context.Background()
conn := db.NewConnWithConfig(config.PostgresConfig{
LogLevel: pgx.LogLevelWarn,
})
defer conn.Close(ctx)
tx := utils.Must1(conn.Begin(ctx))
defer tx.Rollback(ctx)
fmt.Println("Creating HMN project...")
hmn := seedProject(ctx, tx, seedHMN, nil)
utils.Must0(tx.Commit(ctx))
return hmn
}
// Seeds the database with sample data for local dev.
func SampleSeed() {
hmn := BareMinimumSeed()
ctx := context.Background()
conn := db.NewConnWithConfig(config.PostgresConfig{
LogLevel: pgx.LogLevelWarn,
})
defer conn.Close(ctx)
tx := utils.Must1(conn.Begin(ctx))
defer tx.Rollback(ctx)
fmt.Println("Creating admin user (\"admin\"/\"password\")...")
admin := seedUser(ctx, tx, models.User{Username: "admin", Name: "Admin", Email: "admin@handmade.network", IsStaff: true})
fmt.Println("Creating normal users (all with password \"password\")...")
alice := seedUser(ctx, tx, models.User{Username: "alice", Name: "Alice"})
bob := seedUser(ctx, tx, models.User{Username: "bob", Name: "Bob"})
charlie := seedUser(ctx, tx, models.User{Username: "charlie", Name: "Charlie"})
fmt.Println("Creating a spammer...")
spammer := seedUser(ctx, tx, models.User{
Username: "spam",
Status: models.UserStatusConfirmed,
Name: "Hot singletons in your local area",
Bio: "Howdy, everybody I go by Jarva seesharpe from Bangalore. In this way, assuming you need to partake in a shared global instance with me then, at that poi",
})
users := []*models.User{alice, bob, charlie, spammer}
fmt.Println("Creating starter projects...")
hero := seedProject(ctx, tx, seedHandmadeHero, []*models.User{admin})
fourcoder := seedProject(ctx, tx, seed4coder, []*models.User{bob})
for i := 0; i < 5; i++ {
name := fmt.Sprintf("%s %s", lorem.Word(1, 10), lorem.Word(1, 10))
slug := strings.ReplaceAll(strings.ToLower(name), " ", "-")
possibleOwners := []*models.User{alice, bob, charlie}
var owners []*models.User
for ownerIdx, owner := range possibleOwners {
mask := (i % ((1 << len(possibleOwners)) - 1)) + 1
if (1<<ownerIdx)&mask != 0 {
owners = append(owners, owner)
}
}
seedProject(ctx, tx, models.Project{
Slug: slug,
Name: name,
Blurb: lorem.Sentence(6, 16),
Description: lorem.Paragraph(3, 5),
Personal: true,
}, owners)
}
// spam project!
seedProject(ctx, tx, models.Project{
Slug: "spam",
Name: "Cheap abstraction enhancers",
Blurb: "Get higher than ever before...up the ladder of abstraction.",
Description: "Tired of boring details like the actual problem assigned to you? The sky's the limit with these abstraction enhancers, guaranteed to sweep away all those pesky details so you can focus on what matters: \"architecture\".",
Personal: true,
}, []*models.User{spammer})
fmt.Println("Creating some forum threads...")
for i := 0; i < 5; i++ {
for _, project := range []*models.Project{hmn, hero, fourcoder} {
thread := seedThread(ctx, tx, project, models.Thread{})
populateThread(ctx, tx, thread, users, rand.Intn(5)+1)
}
}
// spam-only thread
{
thread := seedThread(ctx, tx, hmn, models.Thread{})
populateThread(ctx, tx, thread, []*models.User{spammer}, 1)
}
fmt.Println("Creating news posts...")
{
// Main site news posts
for i := 0; i < 3; i++ {
thread := seedThread(ctx, tx, hmn, models.Thread{Type: models.ThreadTypeProjectBlogPost})
populateThread(ctx, tx, thread, []*models.User{admin, alice, bob, charlie}, rand.Intn(5)+1)
}
// 4coder
for i := 0; i < 5; i++ {
thread := seedThread(ctx, tx, fourcoder, models.Thread{Type: models.ThreadTypeProjectBlogPost})
populateThread(ctx, tx, thread, []*models.User{bob}, 1)
}
}
// admin := CreateAdminUser("admin", "12345678")
// user := CreateUser("regular_user", "12345678")
// hmnProject := CreateProject("hmn", "Handmade Network")
// Create category
// Create thread
// Create accepted user project
// Create pending user project
// Create showcase items
// Create codelanguages
// Create library and library resources
utils.Must0(tx.Commit(ctx))
}
func seedUser(ctx context.Context, conn db.ConnOrTx, input models.User) *models.User {
user := db.MustQueryOne[models.User](ctx, conn,
`
INSERT INTO hmn_user (
username, password, email,
is_staff,
status,
name, bio, blurb, signature,
darktheme,
showemail, edit_library,
date_joined, registration_ip, avatar_asset_id
)
VALUES (
$1, $2, $3,
$4,
$5,
$6, $7, $8, $9,
TRUE,
$10, FALSE,
'2017-01-01T00:00:00Z', '192.168.2.1', null
)
RETURNING $columns
`,
input.Username, "", utils.OrDefault(input.Email, fmt.Sprintf("%s@example.com", input.Username)),
input.IsStaff,
utils.OrDefault(input.Status, models.UserStatusApproved),
utils.OrDefault(input.Name, randomName()), utils.OrDefault(input.Bio, lorem.Paragraph(0, 2)), utils.OrDefault(input.Blurb, lorem.Sentence(0, 14)), utils.OrDefault(input.Signature, lorem.Sentence(0, 16)),
input.ShowEmail,
)
utils.Must0(auth.SetPassword(ctx, conn, input.Username, "password"))
return user
}
func seedThread(ctx context.Context, tx pgx.Tx, project *models.Project, input models.Thread) *models.Thread {
input.Type = utils.OrDefault(input.Type, models.ThreadTypeForumPost)
var defaultSubforum *int
if input.Type == models.ThreadTypeForumPost {
defaultSubforum = project.ForumID
}
thread, err := db.QueryOne[models.Thread](ctx, tx,
`
INSERT INTO thread (
title,
type, sticky,
project_id, subforum_id,
first_id, last_id
)
VALUES (
$1,
$2, $3,
$4, $5,
$6, $7
)
RETURNING $columns
`,
utils.OrDefault(input.Title, lorem.Sentence(3, 8)),
utils.OrDefault(input.Type, models.ThreadTypeForumPost), false,
project.ID, utils.OrDefault(input.SubforumID, defaultSubforum),
-1, -1,
)
if err != nil {
panic(oops.New(err, "failed to create thread"))
}
return thread
}
func populateThread(ctx context.Context, tx pgx.Tx, thread *models.Thread, users []*models.User, numPosts int) {
var lastPostId int
for i := 0; i < numPosts; i++ {
user := users[i%len(users)]
var replyId *int
if lastPostId != 0 {
if rand.Intn(10) < 3 {
replyId = &lastPostId
}
}
hmndata.CreateNewPost(ctx, tx, thread.ProjectID, thread.ID, thread.Type, user.ID, replyId, lorem.Paragraph(1, 10), "192.168.2.1")
}
}
var latestProjectId int
func seedProject(ctx context.Context, tx pgx.Tx, input models.Project, owners []*models.User) *models.Project {
project := db.MustQueryOne[models.Project](ctx, tx,
`
INSERT INTO project (
id,
slug, name, blurb,
description, descparsed,
color_1, color_2,
featured, personal, lifecycle, hidden,
forum_enabled, blog_enabled,
date_created
)
VALUES (
$1,
$2, $3, $4,
$5, $6,
$7, $8,
$9, $10, $11, $12,
$13, $14,
$15
)
RETURNING $columns
`,
utils.OrDefault(input.ID, latestProjectId+1),
input.Slug, input.Name, input.Blurb,
input.Description, parsing.ParseMarkdown(input.Description, parsing.ForumRealMarkdown),
input.Color1, input.Color2,
input.Featured, input.Personal, utils.OrDefault(input.Lifecycle, models.ProjectLifecycleActive), input.Hidden,
input.ForumEnabled, input.BlogEnabled,
utils.OrDefault(input.DateCreated, time.Now()),
)
latestProjectId = utils.IntMax(latestProjectId, project.ID)
// Create forum (even if unused)
forum := db.MustQueryOne[models.Subforum](ctx, tx,
`
INSERT INTO subforum (
slug, name,
project_id
)
VALUES (
$1, $2,
$3
)
RETURNING $columns
`,
"", project.Name,
project.ID,
)
// Associate forum with project
utils.Must1(tx.Exec(ctx,
`UPDATE project SET forum_id = $1 WHERE id = $2`,
forum.ID, project.ID,
))
project.ForumID = &forum.ID
// Add project owners
for _, owner := range owners {
utils.Must1(tx.Exec(ctx,
`INSERT INTO user_project (user_id, project_id) VALUES ($1, $2)`,
owner.ID, project.ID,
))
}
return project
}
func randomName() string {
return "John Doe" // chosen by fair dice roll. guaranteed to be random.
}
func randomBool() bool {
return rand.Intn(2) == 1
}
var seedHMN = models.Project{
ID: models.HMNProjectID,
Slug: models.HMNProjectSlug,
Name: "Handmade Network",
Blurb: "Changing the way software is written",
Description: `
[project=hero]Originally inspired by Handmade Hero[/project], we're an offshoot of its community, hoping to change the way software is written. To this end we've circulated our [url=https://handmade.network/manifesto]manifesto[/url] and built this website, in the hopes of fostering this community. We invite others to host projects built with same goals in mind and build up or expand their community's reach in our little tree house and hope it proves vibrant soil for the exchange of ideas as well as code.
`,
Color1: "ab4c47", Color2: "a5467d",
Hidden: true,
ForumEnabled: true, BlogEnabled: true,
DateCreated: time.Date(2017, 1, 1, 0, 0, 0, 0, time.UTC),
}
var seedHandmadeHero = models.Project{
Slug: "hero",
Name: "Handmade Hero",
Blurb: "An ongoing project to create a complete, professional-quality game accompanied by videos that explain every single line of its source code.",
Description: `
Handmade Hero is an ongoing project by [Casey Muratori](http://mollyrocket.com/casey) to create a complete, professional-quality game accompanied by videos that explain every single line of its source code. The series began on November 17th, 2014, and is estimated to run for at least 600 episodes. Programming sessions are limited to one hour per weekday so it remains manageable for people who practice coding along with the series at home.
For more information, see the official website at https://handmadehero.org
`,
Color1: "19328a", Color2: "f1f0a2",
Featured: true,
ForumEnabled: true, BlogEnabled: false,
DateCreated: time.Date(2017, 1, 10, 0, 0, 0, 0, time.UTC),
}
var seed4coder = models.Project{
Slug: "4coder",
Name: "4coder",
Blurb: "A programmable, cross platform, IDE template",
Description: `
4coder preview video: https://www.youtube.com/watch?v=Nop5UW2kV3I
4coder differentiates from other editors by focusing on powerful C/C++ customization and extension, and ease of cross platform use. This means that 4coder greatly reduces the cost of creating cross platform development tools such as debuggers, code intelligence systems. It means that tools specialized to your particular needs can be programmed in C/C++ or any language that interfaces with C/C++, which is almost all of them.
In other words, 4coder is attempting to live in a space between an IDE and a power editor such as Emacs or Vim.
Want to try it out? [url=https://4coder.itch.io/4coder]Get your alpha build now[/url]!
`,
Color1: "002107", Color2: "cccccc",
ForumEnabled: true, BlogEnabled: true,
DateCreated: time.Date(2017, 1, 10, 0, 0, 0, 0, time.UTC),
}

View File

@ -1,25 +0,0 @@
Clean this up once we get the website working
---------------------------------------------
TODO: Questionable db tables that we inherited from Django:
* auth_permission
* auth_user_user_permissions
* django_admin_log
* django_content_type
* django_migrations
* django_session
* django_site
TODO: Questionable db tables that we inherited from the old website:
* handmade_blacklistemail
* handmade_blacklisthostname
* handmade_communicationchoice
* handmade_communicationchoicelist
* handmade_communicationsubcategory
* handmade_communicationsubthread
* handmade_kunenapost
* handmade_license
* handmade_license_texts
* handmade_memberextended.joomlaid
* handmade_onetimetoken // Is this for password resets??
* handmade_otherfile
* handmade_project_licenses

View File

@ -0,0 +1,6 @@
package models
type PersistentVar struct {
Name string `db:"name"`
Value string `db:"value"`
}

View File

@ -44,14 +44,10 @@ func (node *SubforumTreeNode) GetLineage() []*Subforum {
}
func GetFullSubforumTree(ctx context.Context, conn *pgxpool.Pool) SubforumTree {
type subforumRow struct {
Subforum Subforum `db:"sf"`
}
rowsSlice, err := db.Query(ctx, conn, subforumRow{},
subforums, err := db.Query[Subforum](ctx, conn,
`
SELECT $columns
FROM
handmade_subforum as sf
FROM subforum
ORDER BY sort, id ASC
`,
)
@ -59,10 +55,9 @@ func GetFullSubforumTree(ctx context.Context, conn *pgxpool.Pool) SubforumTree {
panic(oops.New(err, "failed to fetch subforum tree"))
}
sfTreeMap := make(map[int]*SubforumTreeNode, len(rowsSlice))
for _, row := range rowsSlice {
sf := row.(*subforumRow).Subforum
sfTreeMap[sf.ID] = &SubforumTreeNode{Subforum: sf}
sfTreeMap := make(map[int]*SubforumTreeNode, len(subforums))
for _, sf := range subforums {
sfTreeMap[sf.ID] = &SubforumTreeNode{Subforum: *sf}
}
for _, node := range sfTreeMap {
@ -71,9 +66,8 @@ func GetFullSubforumTree(ctx context.Context, conn *pgxpool.Pool) SubforumTree {
}
}
for _, row := range rowsSlice {
for _, cat := range subforums {
// NOTE(asaf): Doing this in a separate loop over rowsSlice to ensure that Children are in db order.
cat := row.(*subforumRow).Subforum
node := sfTreeMap[cat.ID]
if node.Parent != nil {
node.Parent.Children = append(node.Parent.Children, node)

View File

@ -36,7 +36,6 @@ type User struct {
Blurb string `db:"blurb"`
Signature string `db:"signature"`
AvatarAssetID *uuid.UUID `db:"avatar_asset_id"`
AvatarAsset *Asset `db:"avatar"`
DarkTheme bool `db:"darktheme"`
Timezone string `db:"timezone"`
@ -48,6 +47,9 @@ type User struct {
DiscordDeleteSnippetOnMessageDelete bool `db:"discord_delete_snippet_on_message_delete"`
MarkedAllReadAt time.Time `db:"marked_all_read_at"`
// Non-db fields, to be filled in by fetch helpers
AvatarAsset *Asset
}
func (u *User) BestName() string {

View File

@ -5,6 +5,7 @@ import (
"fmt"
"time"
"git.handmade.network/hmn/hmn/src/jobs"
"github.com/rs/zerolog"
)
@ -109,20 +110,20 @@ type PerfStorage struct {
type PerfCollector struct {
In chan<- RequestPerf
Done <-chan struct{}
Job jobs.Job
RequestCopy chan<- (chan<- PerfStorage)
}
func RunPerfCollector(ctx context.Context) *PerfCollector {
in := make(chan RequestPerf)
done := make(chan struct{})
job := jobs.New()
requestCopy := make(chan (chan<- PerfStorage))
var storage PerfStorage
// TODO(asaf): Load history from file
go func() {
defer close(done)
defer job.Done()
for {
select {
@ -139,7 +140,7 @@ func RunPerfCollector(ctx context.Context) *PerfCollector {
perfCollector := PerfCollector{
In: in,
Done: done,
Job: job,
RequestCopy: requestCopy,
}
return &perfCollector

View File

@ -181,10 +181,12 @@
sizes using CSS grid properties.
*/}}
<div>
<h2>Latest News</h2>
{{ template "timeline_item.html" .NewsPost }}
</div>
{{ with .NewsPost }}
<div>
<h2>Latest News</h2>
{{ template "timeline_item.html" . }}
</div>
{{ end }}
<div class="landing-right">
<h2>Around the Network</h2>
<div class="optionbar mb2">

View File

@ -3,13 +3,13 @@ package twitch
import (
"context"
"encoding/json"
"fmt"
"time"
"git.handmade.network/hmn/hmn/src/config"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/discord"
"git.handmade.network/hmn/hmn/src/hmndata"
"git.handmade.network/hmn/hmn/src/jobs"
"git.handmade.network/hmn/hmn/src/logging"
"git.handmade.network/hmn/hmn/src/models"
"git.handmade.network/hmn/hmn/src/oops"
@ -25,25 +25,23 @@ type twitchNotification struct {
var twitchNotificationChannel chan twitchNotification
var linksChangedChannel chan struct{}
func MonitorTwitchSubscriptions(ctx context.Context, dbConn *pgxpool.Pool) <-chan struct{} {
func MonitorTwitchSubscriptions(ctx context.Context, dbConn *pgxpool.Pool) jobs.Job {
log := logging.ExtractLogger(ctx).With().Str("twitch goroutine", "stream monitor").Logger()
ctx = logging.AttachLoggerToContext(&log, ctx)
if config.Config.Twitch.ClientID == "" {
log.Warn().Msg("No twitch config provided.")
done := make(chan struct{}, 1)
done <- struct{}{}
return done
return jobs.Noop()
}
twitchNotificationChannel = make(chan twitchNotification, 100)
linksChangedChannel = make(chan struct{}, 10)
done := make(chan struct{})
job := jobs.New()
go func() {
defer func() {
log.Info().Msg("Shutting down twitch monitor")
done <- struct{}{}
job.Done()
}()
log.Info().Msg("Running twitch monitor...")
@ -114,7 +112,7 @@ func MonitorTwitchSubscriptions(ctx context.Context, dbConn *pgxpool.Pool) <-cha
}
}()
return done
return job
}
type twitchNotificationType int
@ -328,7 +326,7 @@ func syncWithTwitch(ctx context.Context, dbConn *pgxpool.Pool, updateAll bool) {
}
p.StartBlock("SQL", "Remove untracked streamers")
_, err = tx.Exec(ctx,
`DELETE FROM twitch_streams WHERE twitch_id != ANY($1)`,
`DELETE FROM twitch_stream WHERE twitch_id != ANY($1)`,
allIDs,
)
if err != nil {
@ -363,7 +361,7 @@ func syncWithTwitch(ctx context.Context, dbConn *pgxpool.Pool, updateAll bool) {
p.StartBlock("SQL", "Update stream statuses in db")
for _, status := range statuses {
log.Debug().Interface("Status", status).Msg("Got streamer")
_, err = updateStreamStatusInDB(ctx, tx, &status)
err = updateStreamStatusInDB(ctx, tx, &status)
if err != nil {
log.Error().Err(err).Msg("failed to update twitch stream status")
}
@ -375,19 +373,41 @@ func syncWithTwitch(ctx context.Context, dbConn *pgxpool.Pool, updateAll bool) {
}
stats.NumStreamsChecked += len(usersToUpdate)
log.Info().Interface("Stats", stats).Msg("Twitch sync done")
log.Debug().Msg("Notifying discord")
err = notifyDiscordOfLiveStream(ctx, dbConn)
if err != nil {
log.Error().Err(err).Msg("failed to notify discord")
}
}
func notifyDiscordOfLiveStream(ctx context.Context, dbConn db.ConnOrTx, twitchLogin string, title string) error {
var err error
if config.Config.Discord.StreamsChannelID != "" {
err = discord.SendMessages(ctx, dbConn, discord.MessageToSend{
ChannelID: config.Config.Discord.StreamsChannelID,
Req: discord.CreateMessageRequest{
Content: fmt.Sprintf("%s is live: https://twitch.tv/%s\n> %s", twitchLogin, twitchLogin, title),
},
func notifyDiscordOfLiveStream(ctx context.Context, dbConn db.ConnOrTx) error {
streams, err := db.Query[models.TwitchStream](ctx, dbConn,
`
SELECT $columns
FROM
twitch_stream
ORDER BY started_at ASC
`,
)
if err != nil {
return oops.New(err, "failed to fetch livestreams from db")
}
var streamDetails []hmndata.StreamDetails
for _, s := range streams {
streamDetails = append(streamDetails, hmndata.StreamDetails{
Username: s.Login,
StartTime: s.StartedAt,
Title: s.Title,
})
}
return err
err = discord.UpdateStreamers(ctx, dbConn, streamDetails)
if err != nil {
return oops.New(err, "failed to update discord with livestream info")
}
return nil
}
func processEventSubNotification(ctx context.Context, dbConn db.ConnOrTx, notification *twitchNotification) {
@ -422,41 +442,25 @@ func processEventSubNotification(ctx context.Context, dbConn db.ConnOrTx, notifi
}
log.Debug().Interface("Status", status).Msg("Updating status")
inserted, err := updateStreamStatusInDB(ctx, dbConn, &status)
err = updateStreamStatusInDB(ctx, dbConn, &status)
if err != nil {
log.Error().Err(err).Msg("failed to update twitch stream status")
}
if inserted {
log.Debug().Msg("Notifying discord")
err = notifyDiscordOfLiveStream(ctx, dbConn, status.TwitchLogin, status.Title)
if err != nil {
log.Error().Err(err).Msg("failed to notify discord")
}
log.Debug().Msg("Notifying discord")
err = notifyDiscordOfLiveStream(ctx, dbConn)
if err != nil {
log.Error().Err(err).Msg("failed to notify discord")
}
}
func updateStreamStatusInDB(ctx context.Context, conn db.ConnOrTx, status *streamStatus) (bool, error) {
func updateStreamStatusInDB(ctx context.Context, conn db.ConnOrTx, status *streamStatus) error {
log := logging.ExtractLogger(ctx)
inserted := false
if isStatusRelevant(status) {
log.Debug().Msg("Status relevant")
_, err := db.QueryOne(ctx, conn, models.TwitchStream{},
_, err := conn.Exec(ctx,
`
SELECT $columns
FROM twitch_streams
WHERE twitch_id = $1
`,
status.TwitchID,
)
if err == db.NotFound {
log.Debug().Msg("Inserting new stream")
inserted = true
} else if err != nil {
return false, oops.New(err, "failed to query existing stream")
}
_, err = conn.Exec(ctx,
`
INSERT INTO twitch_streams (twitch_id, twitch_login, title, started_at)
INSERT INTO twitch_stream (twitch_id, twitch_login, title, started_at)
VALUES ($1, $2, $3, $4)
ON CONFLICT (twitch_id) DO UPDATE SET
title = EXCLUDED.title,
@ -468,21 +472,21 @@ func updateStreamStatusInDB(ctx context.Context, conn db.ConnOrTx, status *strea
status.StartedAt,
)
if err != nil {
return false, oops.New(err, "failed to insert twitch streamer into db")
return oops.New(err, "failed to insert twitch streamer into db")
}
} else {
log.Debug().Msg("Stream not relevant")
_, err := conn.Exec(ctx,
`
DELETE FROM twitch_streams WHERE twitch_id = $1
DELETE FROM twitch_stream WHERE twitch_id = $1
`,
status.TwitchID,
)
if err != nil {
return false, oops.New(err, "failed to remove twitch streamer from db")
return oops.New(err, "failed to remove twitch streamer from db")
}
}
return inserted, nil
return nil
}
var RelevantCategories = []string{

View File

@ -10,6 +10,33 @@ import (
"git.handmade.network/hmn/hmn/src/oops"
)
// Returns the provided value, or a default value if the input was zero.
func OrDefault[T comparable](v T, def T) T {
var zero T
if v == zero {
return def
} else {
return v
}
}
// Takes an (error) return and panics if there is an error.
// Helps avoid `if err != nil` in scripts. Use sparingly in real code.
func Must0(err error) {
if err != nil {
panic(err)
}
}
// Takes a (something, error) return and panics if there is an error.
// Helps avoid `if err != nil` in scripts. Use sparingly in real code.
func Must1[T any](v T, err error) T {
if err != nil {
panic(err)
}
return v
}
func IntMin(a, b int) int {
if a < b {
return a

View File

@ -207,11 +207,11 @@ func AdminApprovalQueue(c *RequestContext) ResponseData {
userIds = append(userIds, u.User.ID)
}
userLinks, err := db.Query(c.Context(), c.Conn, models.Link{},
userLinks, err := db.Query[models.Link](c.Context(), c.Conn,
`
SELECT $columns
FROM
handmade_links
link
WHERE
user_id = ANY($1)
ORDER BY ordering ASC
@ -222,8 +222,7 @@ func AdminApprovalQueue(c *RequestContext) ResponseData {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to fetch user links"))
}
for _, ul := range userLinks {
link := ul.(*models.Link)
for _, link := range userLinks {
userData := unapprovedUsers[userIDToDataIdx[*link.UserID]]
userData.UserLinks = append(userData.UserLinks, templates.LinkToTemplate(link))
}
@ -257,18 +256,7 @@ func AdminApprovalQueueSubmit(c *RequestContext) ResponseData {
return RejectRequest(c, "User id can't be parsed")
}
type userQuery struct {
User models.User `db:"auth_user"`
}
u, err := db.QueryOne(c.Context(), c.Conn, userQuery{},
`
SELECT $columns
FROM auth_user
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
WHERE auth_user.id = $1
`,
userId,
)
user, err := hmndata.FetchUser(c.Context(), c.Conn, c.CurrentUser, userId, hmndata.UsersQuery{})
if err != nil {
if errors.Is(err, db.NotFound) {
return RejectRequest(c, "User not found")
@ -276,13 +264,12 @@ func AdminApprovalQueueSubmit(c *RequestContext) ResponseData {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to fetch user"))
}
}
user := u.(*userQuery).User
whatHappened := ""
if action == ApprovalQueueActionApprove {
_, err := c.Conn.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET status = $1
WHERE id = $2
`,
@ -296,7 +283,7 @@ func AdminApprovalQueueSubmit(c *RequestContext) ResponseData {
} else if action == ApprovalQueueActionSpammer {
_, err := c.Conn.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET status = $1
WHERE id = $2
`,
@ -337,16 +324,16 @@ type UnapprovedPost struct {
}
func fetchUnapprovedPosts(c *RequestContext) ([]*UnapprovedPost, error) {
it, err := db.Query(c.Context(), c.Conn, UnapprovedPost{},
posts, err := db.Query[UnapprovedPost](c.Context(), c.Conn,
`
SELECT $columns
FROM
handmade_post AS post
JOIN handmade_project AS project ON post.project_id = project.id
JOIN handmade_thread AS thread ON post.thread_id = thread.id
JOIN handmade_postversion AS ver ON ver.id = post.current_id
JOIN auth_user AS author ON author.id = post.author_id
LEFT JOIN handmade_asset AS author_avatar ON author_avatar.id = author.avatar_asset_id
post
JOIN project ON post.project_id = project.id
JOIN thread ON post.thread_id = thread.id
JOIN post_version AS ver ON ver.id = post.current_id
JOIN hmn_user AS author ON author.id = post.author_id
LEFT JOIN asset AS author_avatar ON author_avatar.id = author.avatar_asset_id
WHERE
NOT thread.deleted
AND NOT post.deleted
@ -358,11 +345,7 @@ func fetchUnapprovedPosts(c *RequestContext) ([]*UnapprovedPost, error) {
if err != nil {
return nil, oops.New(err, "failed to fetch unapproved posts")
}
var res []*UnapprovedPost
for _, iresult := range it {
res = append(res, iresult.(*UnapprovedPost))
}
return res, nil
return posts, nil
}
type UnapprovedProject struct {
@ -372,14 +355,11 @@ type UnapprovedProject struct {
}
func fetchUnapprovedProjects(c *RequestContext) ([]UnapprovedProject, error) {
type unapprovedUser struct {
ID int `db:"id"`
}
it, err := db.Query(c.Context(), c.Conn, unapprovedUser{},
ownerIDs, err := db.QueryScalar[int](c.Context(), c.Conn,
`
SELECT $columns
SELECT id
FROM
auth_user AS u
hmn_user AS u
WHERE
u.status = ANY($1)
`,
@ -388,10 +368,6 @@ func fetchUnapprovedProjects(c *RequestContext) ([]UnapprovedProject, error) {
if err != nil {
return nil, oops.New(err, "failed to fetch unapproved users")
}
ownerIDs := make([]int, 0, len(it))
for _, uid := range it {
ownerIDs = append(ownerIDs, uid.(*unapprovedUser).ID)
}
projects, err := hmndata.FetchProjects(c.Context(), c.Conn, c.CurrentUser, hmndata.ProjectsQuery{
OwnerIDs: ownerIDs,
@ -406,11 +382,11 @@ func fetchUnapprovedProjects(c *RequestContext) ([]UnapprovedProject, error) {
projectIDs = append(projectIDs, p.Project.ID)
}
projectLinks, err := db.Query(c.Context(), c.Conn, models.Link{},
projectLinks, err := db.Query[models.Link](c.Context(), c.Conn,
`
SELECT $columns
FROM
handmade_links AS link
link
WHERE
link.project_id = ANY($1)
ORDER BY link.ordering ASC
@ -425,8 +401,7 @@ func fetchUnapprovedProjects(c *RequestContext) ([]UnapprovedProject, error) {
for idx, proj := range projects {
links := make([]*models.Link, 0, 10) // NOTE(asaf): 10 should be enough for most projects.
for _, l := range projectLinks {
link := l.(*models.Link)
for _, link := range projectLinks {
if *link.ProjectID == proj.Project.ID {
links = append(links, link)
}
@ -455,13 +430,13 @@ func deleteAllPostsForUser(ctx context.Context, conn *pgxpool.Pool, userId int)
ThreadID int `db:"thread.id"`
PostID int `db:"post.id"`
}
it, err := db.Query(ctx, tx, toDelete{},
rows, err := db.Query[toDelete](ctx, tx,
`
SELECT $columns
FROM
handmade_post as post
JOIN handmade_thread AS thread ON post.thread_id = thread.id
JOIN auth_user AS author ON author.id = post.author_id
post as post
JOIN thread ON post.thread_id = thread.id
JOIN hmn_user AS author ON author.id = post.author_id
WHERE author.id = $1
`,
userId,
@ -471,8 +446,7 @@ func deleteAllPostsForUser(ctx context.Context, conn *pgxpool.Pool, userId int)
return oops.New(err, "failed to fetch posts to delete for user")
}
for _, iResult := range it {
row := iResult.(*toDelete)
for _, row := range rows {
hmndata.DeletePost(ctx, tx, row.ThreadID, row.PostID)
}
err = tx.Commit(ctx)
@ -489,12 +463,12 @@ func deleteAllProjectsForUser(ctx context.Context, conn *pgxpool.Pool, userId in
}
defer tx.Rollback(ctx)
toDelete, err := db.Query(ctx, tx, models.Project{},
projectIDsToDelete, err := db.QueryScalar[int](ctx, tx,
`
SELECT $columns
SELECT project.id
FROM
handmade_project AS project
JOIN handmade_user_projects AS up ON up.project_id = project.id
project
JOIN user_project AS up ON up.project_id = project.id
WHERE
up.user_id = $1
`,
@ -504,17 +478,12 @@ func deleteAllProjectsForUser(ctx context.Context, conn *pgxpool.Pool, userId in
return oops.New(err, "failed to fetch user's projects")
}
var projectIds []int
for _, p := range toDelete {
projectIds = append(projectIds, p.(*models.Project).ID)
}
if len(projectIds) > 0 {
if len(projectIDsToDelete) > 0 {
_, err = tx.Exec(ctx,
`
DELETE FROM handmade_project WHERE id = ANY($1)
DELETE FROM project WHERE id = ANY($1)
`,
projectIds,
projectIDsToDelete,
)
if err != nil {
return oops.New(err, "failed to delete user's projects")

View File

@ -19,17 +19,13 @@ func APICheckUsername(c *RequestContext) ResponseData {
requestedUsername := usernameArgs[0]
found = true
c.Perf.StartBlock("SQL", "Fetch user")
type userQuery struct {
User models.User `db:"auth_user"`
}
userResult, err := db.QueryOne(c.Context(), c.Conn, userQuery{},
user, err := db.QueryOne[models.User](c.Context(), c.Conn,
`
SELECT $columns
FROM
auth_user
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
hmn_user
WHERE
LOWER(auth_user.username) = LOWER($1)
LOWER(hmn_user.username) = LOWER($1)
AND status = ANY ($2)
`,
requestedUsername,
@ -43,7 +39,7 @@ func APICheckUsername(c *RequestContext) ResponseData {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to fetch user: %s", requestedUsername))
}
} else {
canonicalUsername = userResult.(*userQuery).User.Username
canonicalUsername = user.Username
}
}

View File

@ -75,14 +75,10 @@ func Login(c *RequestContext) ResponseData {
return res
}
type userQuery struct {
User models.User `db:"auth_user"`
}
userRow, err := db.QueryOne(c.Context(), c.Conn, userQuery{},
user, err := db.QueryOne[models.User](c.Context(), c.Conn,
`
SELECT $columns
FROM auth_user
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
FROM hmn_user
WHERE LOWER(username) = LOWER($1)
`,
username,
@ -94,7 +90,6 @@ func Login(c *RequestContext) ResponseData {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to look up user by username"))
}
}
user := &userRow.(*userQuery).User
success, err := tryLogin(c, user, password)
@ -174,10 +169,10 @@ func RegisterNewUserSubmit(c *RequestContext) ResponseData {
c.Perf.StartBlock("SQL", "Check for existing usernames and emails")
userAlreadyExists := true
_, err := db.QueryInt(c.Context(), c.Conn,
_, err := db.QueryOneScalar[int](c.Context(), c.Conn,
`
SELECT id
FROM auth_user
FROM hmn_user
WHERE LOWER(username) = LOWER($1)
`,
username,
@ -195,10 +190,10 @@ func RegisterNewUserSubmit(c *RequestContext) ResponseData {
}
emailAlreadyExists := true
_, err = db.QueryInt(c.Context(), c.Conn,
_, err = db.QueryOneScalar[int](c.Context(), c.Conn,
`
SELECT id
FROM auth_user
FROM hmn_user
WHERE LOWER(email) = LOWER($1)
`,
emailAddress,
@ -231,7 +226,7 @@ func RegisterNewUserSubmit(c *RequestContext) ResponseData {
var newUserId int
err = tx.QueryRow(c.Context(),
`
INSERT INTO auth_user (username, email, password, date_joined, name, registration_ip)
INSERT INTO hmn_user (username, email, password, date_joined, name, registration_ip)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id
`,
@ -244,7 +239,7 @@ func RegisterNewUserSubmit(c *RequestContext) ResponseData {
ott := models.GenerateToken()
_, err = tx.Exec(c.Context(),
`
INSERT INTO handmade_onetimetoken (token_type, created, expires, token_content, owner_id)
INSERT INTO one_time_token (token_type, created, expires, token_content, owner_id)
VALUES($1, $2, $3, $4, $5)
`,
models.TokenTypeRegistration,
@ -379,7 +374,7 @@ func EmailConfirmationSubmit(c *RequestContext) ResponseData {
_, err = tx.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET status = $1
WHERE id = $2
`,
@ -392,7 +387,7 @@ func EmailConfirmationSubmit(c *RequestContext) ResponseData {
_, err = tx.Exec(c.Context(),
`
DELETE FROM handmade_onetimetoken WHERE id = $1
DELETE FROM one_time_token WHERE id = $1
`,
validationResult.OneTimeToken.ID,
)
@ -454,17 +449,14 @@ func RequestPasswordResetSubmit(c *RequestContext) ResponseData {
return RejectRequest(c, "You must provide a username and an email address.")
}
var user *models.User
c.Perf.StartBlock("SQL", "Fetching user")
type userQuery struct {
User models.User `db:"auth_user"`
User models.User `db:"hmn_user"`
}
userRow, err := db.QueryOne(c.Context(), c.Conn, userQuery{},
user, err := db.QueryOne[models.User](c.Context(), c.Conn,
`
SELECT $columns
FROM auth_user
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
FROM hmn_user
WHERE
LOWER(username) = LOWER($1)
AND LOWER(email) = LOWER($2)
@ -478,16 +470,13 @@ func RequestPasswordResetSubmit(c *RequestContext) ResponseData {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to look up user by username"))
}
}
if userRow != nil {
user = &userRow.(*userQuery).User
}
if user != nil {
c.Perf.StartBlock("SQL", "Fetching existing token")
tokenRow, err := db.QueryOne(c.Context(), c.Conn, models.OneTimeToken{},
resetToken, err := db.QueryOne[models.OneTimeToken](c.Context(), c.Conn,
`
SELECT $columns
FROM handmade_onetimetoken
FROM one_time_token
WHERE
token_type = $1
AND owner_id = $2
@ -501,10 +490,6 @@ func RequestPasswordResetSubmit(c *RequestContext) ResponseData {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to fetch onetimetoken for user"))
}
}
var resetToken *models.OneTimeToken
if tokenRow != nil {
resetToken = tokenRow.(*models.OneTimeToken)
}
now := time.Now()
if resetToken != nil {
@ -512,7 +497,7 @@ func RequestPasswordResetSubmit(c *RequestContext) ResponseData {
c.Perf.StartBlock("SQL", "Deleting expired token")
_, err = c.Conn.Exec(c.Context(),
`
DELETE FROM handmade_onetimetoken
DELETE FROM one_time_token
WHERE id = $1
`,
resetToken.ID,
@ -527,9 +512,9 @@ func RequestPasswordResetSubmit(c *RequestContext) ResponseData {
if resetToken == nil {
c.Perf.StartBlock("SQL", "Creating new token")
tokenRow, err := db.QueryOne(c.Context(), c.Conn, models.OneTimeToken{},
newToken, err := db.QueryOne[models.OneTimeToken](c.Context(), c.Conn,
`
INSERT INTO handmade_onetimetoken (token_type, created, expires, token_content, owner_id)
INSERT INTO one_time_token (token_type, created, expires, token_content, owner_id)
VALUES ($1, $2, $3, $4, $5)
RETURNING $columns
`,
@ -543,7 +528,7 @@ func RequestPasswordResetSubmit(c *RequestContext) ResponseData {
if err != nil {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to create onetimetoken"))
}
resetToken = tokenRow.(*models.OneTimeToken)
resetToken = newToken
err = email.SendPasswordReset(user.Email, user.BestName(), user.Username, resetToken.Content, resetToken.Expires, c.Perf)
if err != nil {
@ -641,7 +626,7 @@ func DoPasswordResetSubmit(c *RequestContext) ResponseData {
tag, err := tx.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET password = $1
WHERE id = $2
`,
@ -655,7 +640,7 @@ func DoPasswordResetSubmit(c *RequestContext) ResponseData {
if validationResult.User.Status == models.UserStatusInactive {
_, err = tx.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET status = $1
WHERE id = $2
`,
@ -669,7 +654,7 @@ func DoPasswordResetSubmit(c *RequestContext) ResponseData {
_, err = tx.Exec(c.Context(),
`
DELETE FROM handmade_onetimetoken
DELETE FROM one_time_token
WHERE id = $1
`,
validationResult.OneTimeToken.ID,
@ -736,7 +721,7 @@ func loginUser(c *RequestContext, user *models.User, responseData *ResponseData)
_, err = tx.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET last_login = $1
WHERE id = $2
`,
@ -784,17 +769,17 @@ func validateUsernameAndToken(c *RequestContext, username string, token string,
c.Perf.StartBlock("SQL", "Check username and token")
defer c.Perf.EndBlock()
type userAndTokenQuery struct {
User models.User `db:"auth_user"`
User models.User `db:"hmn_user"`
OneTimeToken *models.OneTimeToken `db:"onetimetoken"`
}
row, err := db.QueryOne(c.Context(), c.Conn, userAndTokenQuery{},
data, err := db.QueryOne[userAndTokenQuery](c.Context(), c.Conn,
`
SELECT $columns
FROM auth_user
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
LEFT JOIN handmade_onetimetoken AS onetimetoken ON onetimetoken.owner_id = auth_user.id
FROM hmn_user
LEFT JOIN asset AS hmn_user_avatar ON hmn_user_avatar.id = hmn_user.avatar_asset_id
LEFT JOIN one_time_token AS onetimetoken ON onetimetoken.owner_id = hmn_user.id
WHERE
LOWER(auth_user.username) = LOWER($1)
LOWER(hmn_user.username) = LOWER($1)
AND onetimetoken.token_type = $2
`,
username,
@ -807,8 +792,7 @@ func validateUsernameAndToken(c *RequestContext, username string, token string,
return result
}
}
if row != nil {
data := row.(*userAndTokenQuery)
if data != nil {
result.User = &data.User
result.OneTimeToken = data.OneTimeToken
if result.OneTimeToken != nil {

View File

@ -155,7 +155,7 @@ func BlogThread(c *RequestContext) ResponseData {
c.Perf.StartBlock("SQL", "Update TLRI")
_, err := c.Conn.Exec(c.Context(),
`
INSERT INTO handmade_threadlastreadinfo (thread_id, user_id, lastread)
INSERT INTO thread_last_read_info (thread_id, user_id, lastread)
VALUES ($1, $2, $3)
ON CONFLICT (thread_id, user_id) DO UPDATE
SET lastread = EXCLUDED.lastread
@ -248,7 +248,7 @@ func BlogNewThreadSubmit(c *RequestContext) ResponseData {
var threadId int
err = tx.QueryRow(c.Context(),
`
INSERT INTO handmade_thread (title, type, project_id, first_id, last_id)
INSERT INTO thread (title, type, project_id, first_id, last_id)
VALUES ($1, $2, $3, $4, $5)
RETURNING id
`,
@ -360,7 +360,7 @@ func BlogPostEditSubmit(c *RequestContext) ResponseData {
if title != "" {
_, err := tx.Exec(c.Context(),
`
UPDATE handmade_thread SET title = $1 WHERE id = $2
UPDATE thread SET title = $1 WHERE id = $2
`,
title,
post.Thread.ID,
@ -558,10 +558,10 @@ func getCommonBlogData(c *RequestContext) (commonBlogData, bool) {
res.ThreadID = threadId
c.Perf.StartBlock("SQL", "Verify that the thread exists")
threadExists, err := db.QueryBool(c.Context(), c.Conn,
threadExists, err := db.QueryOneScalar[bool](c.Context(), c.Conn,
`
SELECT COUNT(*) > 0
FROM handmade_thread
FROM thread
WHERE
id = $1
AND project_id = $2
@ -586,10 +586,10 @@ func getCommonBlogData(c *RequestContext) (commonBlogData, bool) {
res.PostID = postId
c.Perf.StartBlock("SQL", "Verify that the post exists")
postExists, err := db.QueryBool(c.Context(), c.Conn,
postExists, err := db.QueryOneScalar[bool](c.Context(), c.Conn,
`
SELECT COUNT(*) > 0
FROM handmade_post
FROM post
WHERE
id = $1
AND thread_id = $2

View File

@ -61,7 +61,7 @@ func DiscordOAuthCallback(c *RequestContext) ResponseData {
// Add the user to our database
_, err = c.Conn.Exec(c.Context(),
`
INSERT INTO handmade_discorduser (username, discriminator, access_token, refresh_token, avatar, locale, userid, expiry, hmn_user_id)
INSERT INTO discord_user (username, discriminator, access_token, refresh_token, avatar, locale, userid, expiry, hmn_user_id)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
`,
user.Username,
@ -81,7 +81,7 @@ func DiscordOAuthCallback(c *RequestContext) ResponseData {
if c.CurrentUser.Status == models.UserStatusConfirmed {
_, err = c.Conn.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET status = $1
WHERE id = $2
`,
@ -104,10 +104,10 @@ func DiscordUnlink(c *RequestContext) ResponseData {
}
defer tx.Rollback(c.Context())
iDiscordUser, err := db.QueryOne(c.Context(), tx, models.DiscordUser{},
discordUser, err := db.QueryOne[models.DiscordUser](c.Context(), tx,
`
SELECT $columns
FROM handmade_discorduser
FROM discord_user
WHERE hmn_user_id = $1
`,
c.CurrentUser.ID,
@ -119,11 +119,10 @@ func DiscordUnlink(c *RequestContext) ResponseData {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to get Discord user for unlink"))
}
}
discordUser := iDiscordUser.(*models.DiscordUser)
_, err = tx.Exec(c.Context(),
`
DELETE FROM handmade_discorduser
DELETE FROM discord_user
WHERE id = $1
`,
discordUser.ID,
@ -146,8 +145,8 @@ func DiscordUnlink(c *RequestContext) ResponseData {
}
func DiscordShowcaseBacklog(c *RequestContext) ResponseData {
iduser, err := db.QueryOne(c.Context(), c.Conn, models.DiscordUser{},
`SELECT $columns FROM handmade_discorduser WHERE hmn_user_id = $1`,
duser, err := db.QueryOne[models.DiscordUser](c.Context(), c.Conn,
`SELECT $columns FROM discord_user WHERE hmn_user_id = $1`,
c.CurrentUser.ID,
)
if errors.Is(err, db.NotFound) {
@ -157,16 +156,12 @@ func DiscordShowcaseBacklog(c *RequestContext) ResponseData {
} else if err != nil {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to get discord user"))
}
duser := iduser.(*models.DiscordUser)
type messageIdQuery struct {
MessageID string `db:"msg.id"`
}
iMsgIDs, err := db.Query(c.Context(), c.Conn, messageIdQuery{},
msgIDs, err := db.QueryScalar[string](c.Context(), c.Conn,
`
SELECT $columns
SELECT msg.id
FROM
handmade_discordmessage AS msg
discord_message AS msg
WHERE
msg.user_id = $1
AND msg.channel_id = $2
@ -178,10 +173,6 @@ func DiscordShowcaseBacklog(c *RequestContext) ResponseData {
return c.ErrorResponse(http.StatusInternalServerError, err)
}
var msgIDs []string
for _, imsgId := range iMsgIDs {
msgIDs = append(msgIDs, imsgId.(*messageIdQuery).MessageID)
}
for _, msgID := range msgIDs {
interned, err := discord.FetchInternedMessage(c.Context(), c.Conn, msgID)
if err != nil && !errors.Is(err, db.NotFound) {

View File

@ -223,7 +223,7 @@ func ForumMarkRead(c *RequestContext) ResponseData {
// Mark literally everything as read
_, err := tx.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET marked_all_read_at = NOW()
WHERE id = $1
`,
@ -236,7 +236,7 @@ func ForumMarkRead(c *RequestContext) ResponseData {
// Delete thread unread info
_, err = tx.Exec(c.Context(),
`
DELETE FROM handmade_threadlastreadinfo
DELETE FROM thread_last_read_info
WHERE user_id = $1;
`,
c.CurrentUser.ID,
@ -248,7 +248,7 @@ func ForumMarkRead(c *RequestContext) ResponseData {
// Delete subforum unread info
_, err = tx.Exec(c.Context(),
`
DELETE FROM handmade_subforumlastreadinfo
DELETE FROM subforum_last_read_info
WHERE user_id = $1;
`,
c.CurrentUser.ID,
@ -260,9 +260,9 @@ func ForumMarkRead(c *RequestContext) ResponseData {
c.Perf.StartBlock("SQL", "Update SLRIs")
_, err = tx.Exec(c.Context(),
`
INSERT INTO handmade_subforumlastreadinfo (subforum_id, user_id, lastread)
INSERT INTO subforum_last_read_info (subforum_id, user_id, lastread)
SELECT id, $2, $3
FROM handmade_subforum
FROM subforum
WHERE id = ANY ($1)
ON CONFLICT (subforum_id, user_id) DO UPDATE
SET lastread = EXCLUDED.lastread
@ -279,12 +279,12 @@ func ForumMarkRead(c *RequestContext) ResponseData {
c.Perf.StartBlock("SQL", "Delete TLRIs")
_, err = tx.Exec(c.Context(),
`
DELETE FROM handmade_threadlastreadinfo
DELETE FROM thread_last_read_info
WHERE
user_id = $2
AND thread_id IN (
SELECT id
FROM handmade_thread
FROM thread
WHERE
subforum_id = ANY ($1)
)
@ -402,7 +402,7 @@ func ForumThread(c *RequestContext) ResponseData {
c.Perf.StartBlock("SQL", "Update TLRI")
_, err = c.Conn.Exec(c.Context(),
`
INSERT INTO handmade_threadlastreadinfo (thread_id, user_id, lastread)
INSERT INTO thread_last_read_info (thread_id, user_id, lastread)
VALUES ($1, $2, $3)
ON CONFLICT (thread_id, user_id) DO UPDATE
SET lastread = EXCLUDED.lastread
@ -523,7 +523,7 @@ func ForumNewThreadSubmit(c *RequestContext) ResponseData {
var threadId int
err = tx.QueryRow(c.Context(),
`
INSERT INTO handmade_thread (title, sticky, type, project_id, subforum_id, first_id, last_id)
INSERT INTO thread (title, sticky, type, project_id, subforum_id, first_id, last_id)
VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING id
`,
@ -710,7 +710,7 @@ func ForumPostEditSubmit(c *RequestContext) ResponseData {
if title != "" {
_, err := tx.Exec(c.Context(),
`
UPDATE handmade_thread SET title = $1 WHERE id = $2
UPDATE thread SET title = $1 WHERE id = $2
`,
title,
post.Thread.ID,
@ -936,12 +936,12 @@ func addForumUrlsToPost(urlContext *hmnurl.UrlContext, p *templates.Post, subfor
// Takes a template post and adds information about how many posts the user has made
// on the site.
func addAuthorCountsToPost(ctx context.Context, conn db.ConnOrTx, p *templates.Post) {
numPosts, err := db.QueryInt(ctx, conn,
numPosts, err := db.QueryOneScalar[int](ctx, conn,
`
SELECT COUNT(*)
FROM
handmade_post AS post
JOIN handmade_project AS project ON post.project_id = project.id
post
JOIN project ON post.project_id = project.id
WHERE
post.author_id = $1
AND NOT post.deleted
@ -956,12 +956,12 @@ func addAuthorCountsToPost(ctx context.Context, conn db.ConnOrTx, p *templates.P
p.AuthorNumPosts = numPosts
}
numProjects, err := db.QueryInt(ctx, conn,
numProjects, err := db.QueryOneScalar[int](ctx, conn,
`
SELECT COUNT(*)
FROM
handmade_project AS project
JOIN handmade_user_projects AS uproj ON uproj.project_id = project.id
project
JOIN user_project AS uproj ON uproj.project_id = project.id
WHERE
project.lifecycle = ANY ($1)
AND uproj.user_id = $2

View File

@ -89,10 +89,9 @@ func SaveImageFile(c *RequestContext, dbConn db.ConnOrTx, fileFieldName string,
img.Seek(0, io.SeekStart)
io.Copy(hasher, img) // NOTE(asaf): Writing to hash.Hash never returns an error according to the docs
sha1sum := hasher.Sum(nil)
// TODO(db): Should use insert helper
imageFile, err := db.QueryOne(c.Context(), dbConn, models.ImageFile{},
imageFile, err := db.QueryOne[models.ImageFile](c.Context(), dbConn,
`
INSERT INTO handmade_imagefile (file, size, sha1sum, protected, width, height)
INSERT INTO image_file (file, size, sha1sum, protected, width, height)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING $columns
`,
@ -105,7 +104,7 @@ func SaveImageFile(c *RequestContext, dbConn db.ConnOrTx, fileFieldName string,
}
return SaveImageFileResult{
ImageFile: imageFile.(*models.ImageFile),
ImageFile: imageFile,
}
}

View File

@ -30,10 +30,9 @@ func ParseLinks(text string) []ParsedLink {
return res
}
func LinksToText(links []interface{}) string {
func LinksToText(links []*models.Link) string {
linksText := ""
for _, l := range links {
link := l.(*models.Link)
for _, link := range links {
linksText += fmt.Sprintf("%s %s\n", link.URL, link.Name)
}
return linksText

View File

@ -3,6 +3,8 @@ package website
import (
"math"
"strconv"
"git.handmade.network/hmn/hmn/src/utils"
)
func getPageInfo(
@ -14,7 +16,7 @@ func getPageInfo(
totalPages int,
ok bool,
) {
totalPages = int(math.Ceil(float64(totalItems) / float64(itemsPerPage)))
totalPages = utils.IntMax(1, int(math.Ceil(float64(totalItems)/float64(itemsPerPage))))
ok = true
page = 1

View File

@ -0,0 +1,36 @@
package website
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestGetPageInfo(t *testing.T) {
items := []struct {
name string
pageParam string
totalItems, perPage int
page, totalPages int
ok bool
}{
{"good, no param", "", 85, 10, 1, 9, true},
{"good", "2", 85, 10, 2, 9, true},
{"too big", "10", 85, 10, 0, 0, false},
{"too small", "0", 85, 10, 0, 0, false},
{"pizza", "pizza", 85, 10, 0, 0, false},
{"zero items, no param", "", 0, 10, 1, 1, true}, // should go to page 1
{"zero items, page 1", "1", 0, 10, 1, 1, true},
{"zero items, too big", "2", 0, 10, 0, 0, false},
{"zero items, too small", "0", 0, 10, 0, 0, false},
}
for _, item := range items {
t.Run(item.name, func(t *testing.T) {
page, totalPages, ok := getPageInfo(item.pageParam, item.totalItems, item.perPage)
assert.Equal(t, item.page, page)
assert.Equal(t, item.totalPages, totalPages)
assert.Equal(t, item.ok, ok)
})
}
}

View File

@ -150,7 +150,7 @@ func PodcastEditSubmit(c *RequestContext) ResponseData {
if imageSaveResult.ImageFile != nil {
_, err = tx.Exec(c.Context(),
`
UPDATE handmade_podcast
UPDATE podcast
SET
title = $1,
description = $2,
@ -168,7 +168,7 @@ func PodcastEditSubmit(c *RequestContext) ResponseData {
} else {
_, err = tx.Exec(c.Context(),
`
UPDATE handmade_podcast
UPDATE podcast
SET
title = $1,
description = $2
@ -419,7 +419,7 @@ func PodcastEpisodeSubmit(c *RequestContext) ResponseData {
c.Perf.StartBlock("SQL", "Updating podcast episode")
_, err := c.Conn.Exec(c.Context(),
`
UPDATE handmade_podcastepisode
UPDATE podcast_episode
SET
title = $1,
description = $2,
@ -448,7 +448,7 @@ func PodcastEpisodeSubmit(c *RequestContext) ResponseData {
c.Perf.StartBlock("SQL", "Creating new podcast episode")
_, err := c.Conn.Exec(c.Context(),
`
INSERT INTO handmade_podcastepisode
INSERT INTO podcast_episode
(guid, title, description, description_rendered, audio_filename, duration, pub_date, episode_number, podcast_id)
VALUES
($1, $2, $3, $4, $5, $6, $7, $8, $9)
@ -532,11 +532,12 @@ func FetchPodcast(c *RequestContext, projectId int, fetchEpisodes bool, episodeG
Podcast models.Podcast `db:"podcast"`
ImageFilename string `db:"imagefile.file"`
}
podcastQueryResult, err := db.QueryOne(c.Context(), c.Conn, podcastQuery{},
podcastQueryResult, err := db.QueryOne[podcastQuery](c.Context(), c.Conn,
`
SELECT $columns
FROM handmade_podcast AS podcast
LEFT JOIN handmade_imagefile AS imagefile ON imagefile.id = podcast.image_id
FROM
podcast
LEFT JOIN image_file AS imagefile ON imagefile.id = podcast.image_id
WHERE podcast.project_id = $1
`,
projectId,
@ -549,21 +550,18 @@ func FetchPodcast(c *RequestContext, projectId int, fetchEpisodes bool, episodeG
return result, oops.New(err, "failed to fetch podcast")
}
}
podcast := podcastQueryResult.(*podcastQuery).Podcast
podcastImageFilename := podcastQueryResult.(*podcastQuery).ImageFilename
podcast := podcastQueryResult.Podcast
podcastImageFilename := podcastQueryResult.ImageFilename
result.Podcast = &podcast
result.ImageFile = podcastImageFilename
if fetchEpisodes {
type podcastEpisodeQuery struct {
Episode models.PodcastEpisode `db:"episode"`
}
if episodeGUID == "" {
c.Perf.StartBlock("SQL", "Fetch podcast episodes")
podcastEpisodeQueryResult, err := db.Query(c.Context(), c.Conn, podcastEpisodeQuery{},
episodes, err := db.Query[models.PodcastEpisode](c.Context(), c.Conn,
`
SELECT $columns
FROM handmade_podcastepisode AS episode
FROM podcast_episode AS episode
WHERE episode.podcast_id = $1
ORDER BY episode.season_number DESC, episode.episode_number DESC
`,
@ -573,19 +571,17 @@ func FetchPodcast(c *RequestContext, projectId int, fetchEpisodes bool, episodeG
if err != nil {
return result, oops.New(err, "failed to fetch podcast episodes")
}
for _, episodeRow := range podcastEpisodeQueryResult {
result.Episodes = append(result.Episodes, &episodeRow.(*podcastEpisodeQuery).Episode)
}
result.Episodes = episodes
} else {
guid, err := uuid.Parse(episodeGUID)
if err != nil {
return result, err
}
c.Perf.StartBlock("SQL", "Fetch podcast episode")
podcastEpisodeQueryResult, err := db.QueryOne(c.Context(), c.Conn, podcastEpisodeQuery{},
episode, err := db.QueryOne[models.PodcastEpisode](c.Context(), c.Conn,
`
SELECT $columns
FROM handmade_podcastepisode AS episode
FROM podcast_episode AS episode
WHERE episode.podcast_id = $1 AND episode.guid = $2
`,
podcast.ID,
@ -599,8 +595,7 @@ func FetchPodcast(c *RequestContext, projectId int, fetchEpisodes bool, episodeG
return result, oops.New(err, "failed to fetch podcast episode")
}
}
episode := podcastEpisodeQueryResult.(*podcastEpisodeQuery).Episode
result.Episodes = append(result.Episodes, &episode)
result.Episodes = append(result.Episodes, episode)
}
}

View File

@ -187,17 +187,14 @@ func ProjectHomepage(c *RequestContext) ResponseData {
}
c.Perf.StartBlock("SQL", "Fetching screenshots")
type screenshotQuery struct {
Filename string `db:"screenshot.file"`
}
screenshotQueryResult, err := db.Query(c.Context(), c.Conn, screenshotQuery{},
screenshotFilenames, err := db.QueryScalar[string](c.Context(), c.Conn,
`
SELECT $columns
SELECT screenshot.file
FROM
handmade_imagefile AS screenshot
INNER JOIN handmade_project_screenshots ON screenshot.id = handmade_project_screenshots.imagefile_id
image_file AS screenshot
INNER JOIN project_screenshot ON screenshot.id = project_screenshot.imagefile_id
WHERE
handmade_project_screenshots.project_id = $1
project_screenshot.project_id = $1
`,
c.CurrentProject.ID,
)
@ -207,14 +204,11 @@ func ProjectHomepage(c *RequestContext) ResponseData {
c.Perf.EndBlock()
c.Perf.StartBlock("SQL", "Fetching project links")
type projectLinkQuery struct {
Link models.Link `db:"link"`
}
projectLinkResult, err := db.Query(c.Context(), c.Conn, projectLinkQuery{},
projectLinks, err := db.Query[models.Link](c.Context(), c.Conn,
`
SELECT $columns
FROM
handmade_links as link
link as link
WHERE
link.project_id = $1
ORDER BY link.ordering ASC
@ -237,14 +231,14 @@ func ProjectHomepage(c *RequestContext) ResponseData {
Thread models.Thread `db:"thread"`
Author models.User `db:"author"`
}
postQueryResult, err := db.Query(c.Context(), c.Conn, postQuery{},
posts, err := db.Query[postQuery](c.Context(), c.Conn,
`
SELECT $columns
FROM
handmade_post AS post
INNER JOIN handmade_thread AS thread ON thread.id = post.thread_id
INNER JOIN auth_user AS author ON author.id = post.author_id
LEFT JOIN handmade_asset AS author_avatar ON author_avatar.id = author.avatar_asset_id
post
INNER JOIN thread ON thread.id = post.thread_id
INNER JOIN hmn_user AS author ON author.id = post.author_id
LEFT JOIN asset AS author_avatar ON author_avatar.id = author.avatar_asset_id
WHERE
post.project_id = $1
ORDER BY post.postdate DESC
@ -318,21 +312,21 @@ func ProjectHomepage(c *RequestContext) ResponseData {
}
}
for _, screenshot := range screenshotQueryResult {
templateData.Screenshots = append(templateData.Screenshots, hmnurl.BuildUserFile(screenshot.(*screenshotQuery).Filename))
for _, screenshotFilename := range screenshotFilenames {
templateData.Screenshots = append(templateData.Screenshots, hmnurl.BuildUserFile(screenshotFilename))
}
for _, link := range projectLinkResult {
templateData.ProjectLinks = append(templateData.ProjectLinks, templates.LinkToTemplate(&link.(*projectLinkQuery).Link))
for _, link := range projectLinks {
templateData.ProjectLinks = append(templateData.ProjectLinks, templates.LinkToTemplate(link))
}
for _, post := range postQueryResult {
for _, post := range posts {
templateData.RecentActivity = append(templateData.RecentActivity, PostToTimelineItem(
c.UrlContext,
lineageBuilder,
&post.(*postQuery).Post,
&post.(*postQuery).Thread,
&post.(*postQuery).Author,
&post.Post,
&post.Thread,
&post.Author,
c.Theme,
))
}
@ -445,7 +439,7 @@ func ProjectNewSubmit(c *RequestContext) ResponseData {
var projectId int
err = tx.QueryRow(c.Context(),
`
INSERT INTO handmade_project
INSERT INTO project
(name, blurb, description, descparsed, lifecycle, date_created, all_last_updated)
VALUES
($1, $2, $3, $4, $5, $6, $6)
@ -498,11 +492,11 @@ func ProjectEdit(c *RequestContext) ResponseData {
}
c.Perf.StartBlock("SQL", "Fetching project links")
projectLinkResult, err := db.Query(c.Context(), c.Conn, models.Link{},
projectLinks, err := db.Query[models.Link](c.Context(), c.Conn,
`
SELECT $columns
FROM
handmade_links as link
link as link
WHERE
link.project_id = $1
ORDER BY link.ordering ASC
@ -525,7 +519,7 @@ func ProjectEdit(c *RequestContext) ResponseData {
c.Theme,
)
projectSettings.LinksText = LinksToText(projectLinkResult)
projectSettings.LinksText = LinksToText(projectLinks)
var res ResponseData
res.MustWriteTemplate("project_edit.html", ProjectEditData{
@ -742,7 +736,7 @@ func updateProject(ctx context.Context, tx pgx.Tx, user *models.User, payload *P
_, err := tx.Exec(ctx,
`
UPDATE handmade_project SET
UPDATE project SET
name = $2,
blurb = $3,
description = $4,
@ -769,7 +763,7 @@ func updateProject(ctx context.Context, tx pgx.Tx, user *models.User, payload *P
if user.IsStaff {
_, err = tx.Exec(ctx,
`
UPDATE handmade_project SET
UPDATE project SET
slug = $2,
featured = $3,
personal = $4,
@ -791,7 +785,7 @@ func updateProject(ctx context.Context, tx pgx.Tx, user *models.User, payload *P
if payload.LightLogo.Exists || payload.LightLogo.Remove {
_, err = tx.Exec(ctx,
`
UPDATE handmade_project
UPDATE project
SET
logolight_asset_id = $2
WHERE
@ -808,7 +802,7 @@ func updateProject(ctx context.Context, tx pgx.Tx, user *models.User, payload *P
if payload.DarkLogo.Exists || payload.DarkLogo.Remove {
_, err = tx.Exec(ctx,
`
UPDATE handmade_project
UPDATE project
SET
logodark_asset_id = $2
WHERE
@ -822,14 +816,10 @@ func updateProject(ctx context.Context, tx pgx.Tx, user *models.User, payload *P
}
}
type userQuery struct {
User models.User `db:"auth_user"`
}
ownerRows, err := db.Query(ctx, tx, userQuery{},
owners, err := db.Query[models.User](ctx, tx,
`
SELECT $columns
FROM auth_user
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
FROM hmn_user
WHERE LOWER(username) = ANY ($1)
`,
payload.OwnerUsernames,
@ -840,7 +830,7 @@ func updateProject(ctx context.Context, tx pgx.Tx, user *models.User, payload *P
_, err = tx.Exec(ctx,
`
DELETE FROM handmade_user_projects
DELETE FROM user_project
WHERE project_id = $1
`,
payload.ProjectID,
@ -849,15 +839,15 @@ func updateProject(ctx context.Context, tx pgx.Tx, user *models.User, payload *P
return oops.New(err, "Failed to delete project owners")
}
for _, ownerRow := range ownerRows {
for _, owner := range owners {
_, err = tx.Exec(ctx,
`
INSERT INTO handmade_user_projects
INSERT INTO user_project
(user_id, project_id)
VALUES
($1, $2)
`,
ownerRow.(*userQuery).User.ID,
owner.ID,
payload.ProjectID,
)
if err != nil {
@ -866,14 +856,14 @@ func updateProject(ctx context.Context, tx pgx.Tx, user *models.User, payload *P
}
twitchLoginsPreChange, preErr := hmndata.FetchTwitchLoginsForUserOrProject(ctx, tx, nil, &payload.ProjectID)
_, err = tx.Exec(ctx, `DELETE FROM handmade_links WHERE project_id = $1`, payload.ProjectID)
_, err = tx.Exec(ctx, `DELETE FROM link WHERE project_id = $1`, payload.ProjectID)
if err != nil {
return oops.New(err, "Failed to delete project links")
}
for i, link := range payload.Links {
_, err = tx.Exec(ctx,
`
INSERT INTO handmade_links (name, url, ordering, project_id)
INSERT INTO link (name, url, ordering, project_id)
VALUES ($1, $2, $3, $4)
`,
link.Name,

View File

@ -548,18 +548,9 @@ func getCurrentUserAndSession(c *RequestContext, sessionId string) (*models.User
}
}
type userQuery struct {
User models.User `db:"auth_user"`
}
userRow, err := db.QueryOne(c.Context(), c.Conn, userQuery{},
`
SELECT $columns
FROM auth_user
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
WHERE username = $1
`,
session.Username,
)
user, err := hmndata.FetchUserByUsername(c.Context(), c.Conn, nil, session.Username, hmndata.UsersQuery{
AnyStatus: true,
})
if err != nil {
if errors.Is(err, db.NotFound) {
logging.Debug().Str("username", session.Username).Msg("returning no current user for this request because the user for the session couldn't be found")
@ -568,7 +559,6 @@ func getCurrentUserAndSession(c *RequestContext, sessionId string) (*models.User
return nil, nil, oops.New(err, "failed to get user for session")
}
}
user := &userRow.(*userQuery).User
return user, session, nil
}

View File

@ -16,6 +16,9 @@ func TestLogContextErrors(t *testing.T) {
err1 := errors.New("test error 1")
err2 := errors.New("test error 2")
defer zerolog.SetGlobalLevel(zerolog.GlobalLevel())
zerolog.SetGlobalLevel(zerolog.TraceLevel)
var buf bytes.Buffer
logger := zerolog.New(&buf)
logger.Print("sanity check")

View File

@ -70,7 +70,7 @@ func TwitchEventSubCallback(c *RequestContext) ResponseData {
}
func TwitchDebugPage(c *RequestContext) ResponseData {
streams, err := db.Query(c.Context(), c.Conn, models.TwitchStream{},
streams, err := db.Query[models.TwitchStream](c.Context(), c.Conn,
`
SELECT $columns
FROM
@ -83,8 +83,7 @@ func TwitchDebugPage(c *RequestContext) ResponseData {
}
html := ""
for _, stream := range streams {
s := stream.(*models.TwitchStream)
for _, s := range streams {
html += fmt.Sprintf(`<a href="https://twitch.tv/%s">%s</a>%s<br />`, s.Login, s.Login, s.Title)
}
var res ResponseData

View File

@ -52,22 +52,7 @@ func UserProfile(c *RequestContext) ResponseData {
if c.CurrentUser != nil && strings.ToLower(c.CurrentUser.Username) == username {
profileUser = c.CurrentUser
} else {
c.Perf.StartBlock("SQL", "Fetch user")
type userQuery struct {
User models.User `db:"auth_user"`
}
userResult, err := db.QueryOne(c.Context(), c.Conn, userQuery{},
`
SELECT $columns
FROM
auth_user
LEFT JOIN handmade_asset AS auth_user_avatar ON auth_user_avatar.id = auth_user.avatar_asset_id
WHERE
LOWER(auth_user.username) = $1
`,
username,
)
c.Perf.EndBlock()
user, err := hmndata.FetchUserByUsername(c.Context(), c.Conn, c.CurrentUser, username, hmndata.UsersQuery{})
if err != nil {
if errors.Is(err, db.NotFound) {
return FourOhFour(c)
@ -75,7 +60,7 @@ func UserProfile(c *RequestContext) ResponseData {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to fetch user: %s", username))
}
}
profileUser = &userResult.(*userQuery).User
profileUser = user
}
{
@ -87,14 +72,11 @@ func UserProfile(c *RequestContext) ResponseData {
}
c.Perf.StartBlock("SQL", "Fetch user links")
type userLinkQuery struct {
UserLink models.Link `db:"link"`
}
userLinksSlice, err := db.Query(c.Context(), c.Conn, userLinkQuery{},
userLinks, err := db.Query[models.Link](c.Context(), c.Conn,
`
SELECT $columns
FROM
handmade_links as link
link as link
WHERE
link.user_id = $1
ORDER BY link.ordering ASC
@ -104,9 +86,9 @@ func UserProfile(c *RequestContext) ResponseData {
if err != nil {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to fetch links for user: %s", username))
}
profileUserLinks := make([]templates.Link, 0, len(userLinksSlice))
for _, l := range userLinksSlice {
profileUserLinks = append(profileUserLinks, templates.LinkToTemplate(&l.(*userLinkQuery).UserLink))
profileUserLinks := make([]templates.Link, 0, len(userLinks))
for _, l := range userLinks {
profileUserLinks = append(profileUserLinks, templates.LinkToTemplate(l))
}
c.Perf.EndBlock()
@ -231,10 +213,10 @@ func UserSettings(c *RequestContext) ResponseData {
DiscordShowcaseBacklogUrl string
}
links, err := db.Query(c.Context(), c.Conn, models.Link{},
links, err := db.Query[models.Link](c.Context(), c.Conn,
`
SELECT $columns
FROM handmade_links
FROM link
WHERE user_id = $1
ORDER BY ordering
`,
@ -248,10 +230,10 @@ func UserSettings(c *RequestContext) ResponseData {
var tduser *templates.DiscordUser
var numUnsavedMessages int
iduser, err := db.QueryOne(c.Context(), c.Conn, models.DiscordUser{},
duser, err := db.QueryOne[models.DiscordUser](c.Context(), c.Conn,
`
SELECT $columns
FROM handmade_discorduser
FROM discord_user
WHERE hmn_user_id = $1
`,
c.CurrentUser.ID,
@ -261,16 +243,15 @@ func UserSettings(c *RequestContext) ResponseData {
} else if err != nil {
return c.ErrorResponse(http.StatusInternalServerError, oops.New(err, "failed to fetch user's Discord account"))
} else {
duser := iduser.(*models.DiscordUser)
tmp := templates.DiscordUserToTemplate(duser)
tduser = &tmp
numUnsavedMessages, err = db.QueryInt(c.Context(), c.Conn,
numUnsavedMessages, err = db.QueryOneScalar[int](c.Context(), c.Conn,
`
SELECT COUNT(*)
FROM
handmade_discordmessage AS msg
LEFT JOIN handmade_discordmessagecontent AS c ON c.message_id = msg.id
discord_message AS msg
LEFT JOIN discord_message_content AS c ON c.message_id = msg.id
WHERE
msg.user_id = $1
AND msg.channel_id = $2
@ -349,7 +330,7 @@ func UserSettingsSave(c *RequestContext) ResponseData {
_, err = tx.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET
name = $2,
email = $3,
@ -382,14 +363,14 @@ func UserSettingsSave(c *RequestContext) ResponseData {
twitchLoginsPreChange, preErr := hmndata.FetchTwitchLoginsForUserOrProject(c.Context(), tx, &c.CurrentUser.ID, nil)
linksText := form.Get("links")
links := ParseLinks(linksText)
_, err = tx.Exec(c.Context(), `DELETE FROM handmade_links WHERE user_id = $1`, c.CurrentUser.ID)
_, err = tx.Exec(c.Context(), `DELETE FROM link WHERE user_id = $1`, c.CurrentUser.ID)
if err != nil {
c.Logger.Warn().Err(err).Msg("failed to delete old links")
} else {
for i, link := range links {
_, err := tx.Exec(c.Context(),
`
INSERT INTO handmade_links (name, url, ordering, user_id)
INSERT INTO link (name, url, ordering, user_id)
VALUES ($1, $2, $3, $4)
`,
link.Name,
@ -442,7 +423,7 @@ func UserSettingsSave(c *RequestContext) ResponseData {
if newAvatar.Exists || newAvatar.Remove {
_, err := tx.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET
avatar_asset_id = $2
WHERE
@ -493,7 +474,7 @@ func UserProfileAdminSetStatus(c *RequestContext) ResponseData {
_, err = c.Conn.Exec(c.Context(),
`
UPDATE auth_user
UPDATE hmn_user
SET status = $1
WHERE id = $2
`,

View File

@ -14,6 +14,8 @@ import (
"git.handmade.network/hmn/hmn/src/config"
"git.handmade.network/hmn/hmn/src/db"
"git.handmade.network/hmn/hmn/src/discord"
"git.handmade.network/hmn/hmn/src/hmns3"
"git.handmade.network/hmn/hmn/src/jobs"
"git.handmade.network/hmn/hmn/src/logging"
"git.handmade.network/hmn/hmn/src/perf"
"git.handmade.network/hmn/hmn/src/templates"
@ -33,7 +35,7 @@ var WebsiteCommand = &cobra.Command{
backgroundJobContext, cancelBackgroundJobs := context.WithCancel(context.Background())
longRequestContext, cancelLongRequests := context.WithCancel(context.Background())
conn := db.NewConnPool(config.Config.Postgres.MinConn, config.Config.Postgres.MaxConn)
conn := db.NewConnPool()
perfCollector := perf.RunPerfCollector(backgroundJobContext)
server := http.Server{
@ -41,13 +43,14 @@ var WebsiteCommand = &cobra.Command{
Handler: NewWebsiteRoutes(longRequestContext, conn),
}
backgroundJobsDone := zipJobs(
backgroundJobsDone := jobs.Zip(
auth.PeriodicallyDeleteExpiredSessions(backgroundJobContext, conn),
auth.PeriodicallyDeleteInactiveUsers(backgroundJobContext, conn),
perfCollector.Done,
perfCollector.Job,
discord.RunDiscordBot(backgroundJobContext, conn),
discord.RunHistoryWatcher(backgroundJobContext, conn),
twitch.MonitorTwitchSubscriptions(backgroundJobContext, conn),
hmns3.StartServer(backgroundJobContext),
)
signals := make(chan os.Signal, 1)
@ -81,17 +84,6 @@ var WebsiteCommand = &cobra.Command{
logging.Error().Err(serverErr).Msg("Server shut down unexpectedly")
}
<-backgroundJobsDone
<-backgroundJobsDone.C
},
}
func zipJobs(cs ...<-chan struct{}) <-chan struct{} {
out := make(chan struct{})
go func() {
for _, c := range cs {
<-c
}
close(out)
}()
return out
}