file upload endpoints #69
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
majestrate
force-pushed
the
file-upload-2022-02-23
branch
from
af17900 to
022b82f
Compare
jagerman
reviewed
Feb 25, 2022
jagerman
reviewed
Feb 25, 2022
majestrate
force-pushed
the
file-upload-2022-02-23
branch
from
9bcb943 to
c259466
Compare
|
One other thing: we should clean up the |
majestrate
force-pushed
the
file-upload-2022-02-23
branch
2 times, most recently
from
e22b90f to
3b13c31
Compare
jagerman
reviewed
Feb 28, 2022
jagerman
reviewed
Feb 28, 2022
|
#77 should be ready for merge or cherry-picking now, which adds the |
majestrate
force-pushed
the
file-upload-2022-02-23
branch
4 times, most recently
from
f606123 to
6e2a637
Compare
jagerman
force-pushed
the
file-upload-2022-02-23
branch
from
9c337d2 to
e68b026
Compare
jagerman
force-pushed
the
file-upload-2022-02-23
branch
from
b5285f6 to
6ac7266
Compare
jagerman
approved these changes
Mar 14, 2022
Makes the filename not required (we just put in a null if not provided) for upload or download.
send_file() has optimizations that won't work through an onion request (in particular, we can't extract the response body from such a response, but we need that for onion requests). This changes it to build a flask.Response that streams it via Python file iteration instead, which is a bit less efficient, but allows extracting the body where needed for onion request handling. (It should still stream, rather than slam everything into memory, when invoked directly).
We return some valid non-200 responses (like 201, 304) that shouldn't be producing log spam.
Changes the file upload to only strip bad characters from the filename we use to generate the path on disk, but not the suggested filename we store in the database and return to the user. Also changes the test suite to interpret the random byte filename as latin1 so that it ends up as utf-8 for flask. Otherwise flask does the latin1 conversion itself, and returns utf-8 back, but that breaks the test (because what we get back has gone through an unexpected latin1 -> utf-8 encoding).
In practice it is probably already null, but in theory you could upload a post attachment and then pin that.
operator precedence in comparison fixup
1. Flask's test client is doing something strange when setting a header property starting with \\. Just bypass it entirely at set the header ourself using filename*=UTF-8''... encoding (flask understands this, but doesn't produce it). 2. Make random filename tests deterministic (to avoid intermittent failures), and increase the count to 500 and length to 32 (instead of 128 and 16) so that we're (hopefully) hitting more weird cases. 3. Add a specific test with leading backslashes in the filename because that specifically was causing issues with flask (see point 1). 4. Replace \0 and / with unicode replacement characters in the uploaded filename because those aren't valid filename characters *anywhere*. 5. Do the expected local path filename cleanup and verify that we stored at the expected path (even for utf8 or unsafe filenames).
It wasn't obvious how seconds()/minutes()/etc. worked (I got it wrong the first time I tried it), so rename it inside a `from_now` class that makes it more self-descriptive.
If called after `.room` had been accessed it would return None instead of the id.
Add tests for cross-room file references. We were correctly handling not allowing cross-room references of post attachments, but cross-room references of the room image was allowed (now fixed).
jagerman
force-pushed
the
file-upload-2022-02-23
branch
from
6ac7266 to
27ce595
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
add room file upload and room file fetching endpoints with unit tests
Fixes #65 #80