1
0
Fork 0
forked from mia/0x0
No-bullshit file hosting and URL shortening service
Find a file
Emi Simpson af4b3b06c0
Add support for expiring files
SUPPLEMENTALLY:
- Add an `expiration` field to the `file` table of the database
- Produce a migration for the above change
- Overhaul the cleanup script, and integrate into fhost.py
  (now run using FLASK_APP=fhost flask prune)
- Replace the old cleanup script with a deprecation notice
- Add information about how to expire files to the index
- Update the README with information about the new script

Squashed commits:

Add a note explaining that expired files aren't immediately removed

Show correct times on the index page graph

Improve the migration script, removing the need for --legacy

Use automap in place of an explicit file map in migration

Remove vestigial `touch()`

Don't crash when upgrading a fresh database

Remove vestigial warning about legacy files

More efficiently filter to unexpired files when migrating

https://git.0x0.st/mia/0x0/pulls/72#issuecomment-224

Coalesce updates to the database during migration

https://git.0x0.st/mia/0x0/pulls/72#issuecomment-226

Remove vestigial database model

https://git.0x0.st/mia/0x0/pulls/72#issuecomment-261

prune:  Stream expired files from the database

(as opposed to collecting them all first)

config.example.py:  Add min & max expiration + description
2022-11-29 13:09:26 +01:00
instance Add support for expiring files 2022-11-29 13:09:26 +01:00
migrations Add support for expiring files 2022-11-29 13:09:26 +01:00
nsfw_model Add NSFW detection 2017-10-27 05:28:45 +02:00
templates Add support for expiring files 2022-11-29 13:09:26 +01:00
tests Add tests 2021-05-23 19:13:51 +02:00
.gitignore Support instance config 2021-05-23 19:13:51 +02:00
.mailmap Add mailmap 2021-05-23 19:13:51 +02:00
cleanup.py Add support for expiring files 2022-11-29 13:09:26 +01:00
fhost.py Add support for expiring files 2022-11-29 13:09:26 +01:00
LICENSE Change license to EUPL 2020-11-03 04:01:30 +01:00
nsfw_detect.py Fix some flake8 errors in cleanup and nsfw_detect 2021-08-06 15:19:08 -04:00
pyproject.toml Add tests 2021-05-23 19:13:51 +02:00
README.rst Add support for expiring files 2022-11-29 13:09:26 +01:00
requirements.txt remove short_url and add in-tree URLencoder (#53) 2021-12-01 13:25:33 +01:00

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

The Null Pointer
================

This is a no-bullshit file hosting and URL shortening service that also runs
`0x0.st <https://0x0.st>`_. Use with uWSGI.

Configuration
-------------

To configure 0x0, copy ``instance/config.example.py`` to ``instance/config.py``, then edit
it.   Resonable defaults are set, but there's a couple options you'll need to change
before running 0x0 for the first time.

By default, the configuration is stored in the Flask instance directory.
Normally, this is in `./instance`, but it might be different for your system.
For details, see
`the Flask documentation <https://flask.palletsprojects.com/en/2.0.x/config/#instance-folders>`_.

To customize the home and error pages, simply create a ``templates`` directory
in your instance directory and copy any templates you want to modify there.

If you are running nginx, you should use the ``X-Accel-Redirect`` header.
To make it work, include this in your nginx configs ``server`` block::

    location /up {
        internal;
    }

where ``/up`` is whatever youve configured as ``FHOST_STORAGE_PATH``.

For all other servers, set ``FHOST_USE_X_ACCEL_REDIRECT`` to ``False`` and
``USE_X_SENDFILE`` to ``True``, assuming your server supports this.
Otherwise, Flask will serve the file with chunked encoding, which has several
downsides, one of them being that range requests will not work. This is a
problem for example when streaming media files: It wont be possible to seek,
and some ISOBMFF (MP4) files will not play at all.

To make files expire, simply create a cronjob that runs ``FLASK_APP=fhost
flask prune`` every now and then.

Before running the service for the first time, run ``FLASK_APP=fhost flask db upgrade``.


NSFW Detection
--------------

0x0 supports classification of NSFW content via Yahoos open_nsfw Caffe
neural network model. This works for images and video files and requires
the following:

* Caffe Python module (built for Python 3)
* ``ffmpegthumbnailer`` executable in ``$PATH``


Network Security Considerations
-------------------------------

Keep in mind that 0x0 can fetch files from URLs. This includes your local
network! You should take precautions so that this feature cannot be abused.
0x0 does not (yet) have a way to filter remote URLs, but on Linux, you can
use firewall rules and/or namespaces. This is less error-prone anyway.

For instance, if you are using the excellent `FireHOL <https://firehol.org/>`_,
its very easy to create a group on your system and use it as a condition
in your firewall rules. You would then run the application server under that
group.