Commit graph

53 commits

Author SHA1 Message Date
Mia Herkt
45a414c5ee
Implement request filters
This moves preexisting blacklists to the database, and adds the
following filter types:

    * IP address
    * IP network
    * MIME type
    * User agent

In addition, IP address handling is now done with the ipaddress
module.
2024-09-27 18:30:28 +02:00
Mia Herkt
6393538333
Replace NSFW detector implementation 2024-09-27 06:34:14 +02:00
polina4096
8a912e8744
Fix remote URL content length check off-by-one
Fixes #85
2023-06-04 06:35:11 +02:00
Mia Herkt
3d1facaec3
Store user agent with files
Needed for moderation.
2023-03-29 07:36:49 +02:00
Mia Herkt
e00866f5e4
URL: Explicitly set upper-case table name
Looks like recent SQLAlchemy/Alembic chose to lower-case it by
default. Try not to break existing schemas.
2023-03-29 07:19:47 +02:00
jonas-w
3950f6e8eb
fix 500 error when file extension could not be guessed
when a file without an extension was uploaded
and the mimetypes.guess_extension returned None
because there is no official file extension
for that mimetype a NoneType was subscripted
which yielded a 500 http error
2023-01-15 20:36:39 +01:00
Mia Herkt
f76dbef82f
Fix NSFW detection 2022-12-17 02:32:51 +01:00
Mia Herkt
57c4b6853f
Prevent unreasonably long MIME types 2022-12-13 23:41:12 +01:00
Mia Herkt
d5763a9854
File: Fix 404 case with secret URLs 2022-12-13 23:17:56 +01:00
Mia Herkt
aaf0e4492a
Record file sizes in db
Moderation interface is going to use this.
2022-12-13 23:04:48 +01:00
Mia Herkt
6055a50948
File: Add is_nsfw property 2022-12-13 21:51:39 +01:00
Mia Herkt
a904922cbd
Add support for ClamAV 2022-12-12 07:35:05 +01:00
Mia Herkt
0b80a62f80
Add support for “secret” file URLs
Closes #47
2022-12-01 02:49:28 +01:00
Mia Herkt
ed84d3752c
Fix 500 on invalid paths 2022-12-01 01:26:32 +01:00
Mia Herkt
7661216bc0
Fix handling double file name extensions
Long names would get truncated at the end, causing problems
including unresolvable file URLs. Example with default settings:
    .package.lst → .package.

Fixes #61
2022-12-01 01:19:05 +01:00
Mia Herkt
9214bb4832
Add X-Expires to file response headers
Tells clients when files will expire, in milliseconds since Unix epoch.

Closes #50.
2022-11-30 02:30:52 +01:00
Mia Herkt
e168534258
Allow changing expiration date 2022-11-30 02:19:29 +01:00
Mia Herkt
a182b6199b
Allow management operations like deleting files
This introduces the X-Token header field in the response of newly
uploaded files as a simple way for users to manage their own files.

It does not need to be particularly secure.
2022-11-30 02:19:29 +01:00
Emi Simpson
af4b3b06c0
Add support for expiring files
SUPPLEMENTALLY:
- Add an `expiration` field to the `file` table of the database
- Produce a migration for the above change
- Overhaul the cleanup script, and integrate into fhost.py
  (now run using FLASK_APP=fhost flask prune)
- Replace the old cleanup script with a deprecation notice
- Add information about how to expire files to the index
- Update the README with information about the new script

Squashed commits:

Add a note explaining that expired files aren't immediately removed

Show correct times on the index page graph

Improve the migration script, removing the need for --legacy

Use automap in place of an explicit file map in migration

Remove vestigial `touch()`

Don't crash when upgrading a fresh database

Remove vestigial warning about legacy files

More efficiently filter to unexpired files when migrating

https://git.0x0.st/mia/0x0/pulls/72#issuecomment-224

Coalesce updates to the database during migration

https://git.0x0.st/mia/0x0/pulls/72#issuecomment-226

Remove vestigial database model

https://git.0x0.st/mia/0x0/pulls/72#issuecomment-261

prune:  Stream expired files from the database

(as opposed to collecting them all first)

config.example.py:  Add min & max expiration + description
2022-11-29 13:09:26 +01:00
Mia Herkt
c7a728ce84
Fix non-seekable file handles
Closes #59
2022-08-11 05:49:46 +02:00
Mia Herkt
5216e9ebaf
Open upload blacklist in text mode
This wasn’t working since Flask opens files in bin mode by default.
2022-08-01 18:26:35 +02:00
Alexey Sakovets
0db6e4b895 Fix mime splitting
mime[:mime.find(";")]   will remove last character if mime does not contain ";".
Use mime.split(";") instead.
2022-01-01 23:46:41 +03:00
mia
9c4a0fd5a6 remove short_url and add in-tree URLencoder (#53)
This PR removes the short_url dependency as per issue #41.
This implementation is pretty much the same as in short_url except I've rewritten the enbase() function to be iterative instead of recursive.
The only functions of the class are enbase() and debase() since those were the only functions being used by fhost.

Co-authored-by: 7415963987456321 <hrs70@hi.is>
Reviewed-on: https://git.0x0.st/mia/0x0/pulls/53
Co-authored-by: mia <mia@0x0.st>
Co-committed-by: mia <mia@0x0.st>
2021-12-01 13:25:33 +01:00
Mia Herkt
47ff3a1152
Support instance templates 2021-05-23 19:13:51 +02:00
Mia Herkt
3bdbab96c1
Support client-defined names in file URL routing
This allows requests like /j4Tf.bin/myfilename.dat to be served without
having to rewrite URLs at the frontend server.
2021-05-23 19:13:51 +02:00
Mia Herkt
73045dc5e5
Move storage to classes, support double suffixes 2021-05-23 19:13:51 +02:00
Mia Herkt
cd083a7f83
Use pathlib 2021-05-23 19:13:51 +02:00
Mia Herkt
493119fc68
Drop Flask-Manager 2021-05-23 19:13:51 +02:00
Mia Herkt
59a0439c1a
Drop dump_urls endpoint
who cares lol
(originally added for issue #1 but they never archived 0x0)
2021-05-23 19:13:51 +02:00
Mia Herkt
d815d6855f
Use template responses 2021-05-23 19:13:51 +02:00
Mia Herkt
5db62c2dbf
Support instance config 2021-05-23 19:13:51 +02:00
Mia Herkt
a52127f0c3
Add text/plain exception for .log name extension
Fixes #29
2020-11-03 04:02:13 +01:00
Mia Herkt
a4d496d42d
Change license to EUPL 2020-11-03 04:01:30 +01:00
Martin Herkt
cfee463780
Disable NSFW detection by default
See #11
2018-12-22 19:50:32 +01:00
Martin Herkt
b1c79e582b
add extension override for audio/flac 2018-01-09 08:26:11 +01:00
Martin Herkt
4b8c9eb758
Add extension override for text/x-diff 2017-12-22 21:41:51 +01:00
Martin Herkt
0b288d9a94
Add command to delete all files uploaded by address 2017-11-08 11:29:02 +01:00
Martin Herkt
5608c72e03
Fix indent error in querybl 2017-11-08 11:20:46 +01:00
Martin Herkt
b2d830e2aa
store_url: only accept identity content encoding
Some servers (like IPFS gateways) will use chunked transfer encoding on
anything but identity content encoding. Also, probably fix a potential
zip bomb vulnerability.
2017-10-30 05:36:03 +01:00
Martin Herkt
04b46bd01a
Tweak NSFW threshold
Of course it’ll have a few false positives, but this seems to work well
with the 0x0.st dataset.
2017-10-27 09:38:19 +02:00
Martin Herkt
c3c29ab500
Faster query commands, add removed query filter 2017-10-27 08:53:19 +02:00
Martin Herkt
ebfbc6b153
Add NSFW filter to address query commands 2017-10-27 08:36:36 +02:00
Martin Herkt
7bbeb2d144
Add NSFW detection 2017-10-27 05:28:45 +02:00
Martin Herkt
def5d9802f Add querybl command to query uploads by addresses in blacklist 2017-10-27 02:21:33 +02:00
Martin Herkt
2e509a1faf
Fix try/except syntax 2017-03-27 22:18:38 +02:00
Martin Herkt
462555d532
Add support for upload IP blacklists
The format is one address per line, with # used for comments.
2017-02-03 04:10:58 +01:00
Martin Herkt
714de58180
add more URL validation
Turns out ShareX users and shell script authors are fucking retarded.
2017-01-01 21:20:40 +01:00
Martin Herkt
6bc7690711
add BEACON dump with start index for urlte.am
fixes #1
2017-01-01 21:20:39 +01:00
Martin Herkt
e974834422
do not shorten our host URLs
Turns out ShareX users are fucking retarded.
2017-01-01 20:26:35 +01:00
Martin Herkt
0c5d8690cc
use function to format host URL 2017-01-01 20:26:09 +01:00