Commit graph

5932 commits

Author SHA1 Message Date
Eugen Rochko
61ed133fea
Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
りんすき
c1e77b56a9 fix #6523 (#6524) 2018-02-21 19:33:23 +01:00
takayamaki
f69d7cb43b fix purge_removed_accounts task should suspend account before delete it (#6521) 2018-02-21 16:30:46 +01:00
Eugen Rochko
a7171af0a3
Fix avatar and header issues by using custom geometry detector (#6515)
* Fix avatar and header issues by using custom geometry detector

Revert a part of #6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.

One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.

Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.

* Only invoke convert if dimension or format needs to be changed
2018-02-21 03:40:12 +01:00
Moritz Heiber
a4fd4ad1d5 Fix build error for missing variable interpolation in chown instruction (#6519) 2018-02-20 19:11:36 +01:00
Eugen Rochko
02856073f7
Fix #6509: Use pull queue for chewy jobs (#6513) 2018-02-20 17:25:16 +01:00
Eugen Rochko
be9bab171d
Set Docker permissions during the build process (#6514)
* Set Docker permissions during the build process

* Remove docker_entrypoint.sh and use COPY with chown
2018-02-20 17:25:01 +01:00
Jenkins
544543e40a Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-20 16:17:11 +00:00
abcang
7124881273 Improve performance of feed_manager_spec (#6517) 2018-02-20 16:50:12 +01:00
Jenkins
1445ba1703 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-19 15:17:14 +00:00
Eugen Rochko
66105929e0
Don't resize avatars/headers if their dimensions are already right (#6508)
Also don't apply "-quality 80" option which is probably the reason
for slight color differences between original and remote image
(because it would apply it twice, once on original instance, and
again on the receiving instance)
2018-02-19 16:06:12 +01:00
Eugen Rochko
cbb69d41f6
Fix media spoiler design (#6507)
- 4px rounded corners on media attachments
- Better colors/contrast for CW/media spoiler on public pages
- Fix vertical alignment of "Show more" button
- Fix layout jump when unhiding standalone media
2018-02-19 02:39:18 +01:00
Konrad Pozniak
bb26cdda24 add parameter locked to /api/v1/update_credentials (#6506) 2018-02-18 22:57:53 +01:00
Kazushige Tominaga
78936461d7 Added fetch_remote_status_service call spec case actibitypub (#6500)
* Added #link_header spec

* Added #call spec

* Delete spec of private methods

* Added call test case activitypub
2018-02-18 16:34:03 +01:00
HellPie
bc6751ecce Remove outline from body window (Fixes #6501) (#6502) 2018-02-18 16:32:58 +01:00
Akihiko Odaki
51869f2a8c Remove unnecessary g++ configuration (#6499) 2018-02-18 16:32:17 +01:00
Eugen Rochko
cba2897108
Cache relationships in API (#6482)
* Cache relationships in API

* Fetch relationships for search results in UI

* Only save one account's maps in each cache item
2018-02-18 03:14:46 +01:00
Akihiko Odaki
9b8a448477 Isolate each specs for cache store (#6450)
The cache store is explicitly used by some specs, but they were not
isolated and therefore not reliable. This fixes the issue by clearing
the cache after each specs.
2018-02-17 22:35:05 +01:00
Eugen Rochko
a71af98401
Push discovered status through streaming API within a time window (#6484)
Time window of 6 hours
2018-02-17 14:28:48 +01:00
Akihiko Odaki
a7c50c7aba Limit the languages used for notification mailer test (#6487)
Some available languages lack translations for notification mails. Now it
tests for two languages which is certain to have required translations:
German and English.

German is the language the current project owner, Eugen Rochko speaks, and
providing English translations for new messages is de facto mandatory.
2018-02-17 14:27:51 +01:00
David Yip
3d033a4687
Merge remote-tracking branch 'tootsuite/master' into merge-upstream
Conflicts:
	Gemfile
	config/locales/simple_form.pl.yml
2018-02-17 00:02:37 -06:00
Simó Albert i Beltran
c770b503c0 Fix Spanish translation of remote_follow acct (#6475) 2018-02-17 03:53:38 +09:00
Simó Albert i Beltran
ffdf0f2ff6 Fix Catalan translation of remote_follow acct (#6476) 2018-02-16 19:15:56 +01:00
Daniel Hunsaker
eb3262b941 [Nanobox] Fix backups for file storage (#6483) 2018-02-16 17:10:01 +01:00
Eugen Rochko
9dbae6e8a1
Save video metadata and improve video OpenGraph tags (#6481)
* Save metadata from video attachments, put correct dimensions into OG tags

* Add twitter:player for videos

* Fix code style and test
2018-02-16 07:22:20 +01:00
Eugen Rochko
1122579216
Do not hide NSFW media/CW'd text in OpenGraph tags (#6479)
Reasoning: HTML title tag affects everyone. But OpenGraph only affects
when somebody is deliberately sharing the content, usually in an
environment where such content is expected. Hiding the content in
OpenGraph tags results in deceitful previews which inhibit the
shareability of the post.

Example: Somebody writes a clever post about politics but kindly
puts a "uspol" content warning on it. Mastodon users are thankful,
but sharing this post on another platform results in non-Mastodon
users believing the entire contents of the post is "uspol" and not
clicking through/reading and re-sharing.
2018-02-16 04:40:53 +01:00
Eugen Rochko
478ca39e5e
After click to embed video, autoplay it (#6480) 2018-02-15 23:05:12 +01:00
Eugen Rochko
f7765acf9d
Fix #5173: Click card to embed external content (#6471) 2018-02-15 07:04:28 +01:00
abcang
ecdac9017e Fix media button type (#6478) 2018-02-15 04:40:42 +01:00
Marcin Mikołajczak
ba8ec4eed6 i18n: Update Polish translation (#6470)
Signed-off-by: Marcin Mikołajczak <me@m4sk.in>
2018-02-14 07:55:45 +09:00
Jenkins
e9052ceaaf Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-11 23:17:11 +00:00
Daniel King
6ef3874b2e Fix URLs incorrectly having trailing hyphen removed (#6465)
In cases where a URL has a trailing hyphen the FetchLinkCardService incorrectly removes the hyphen when it is parsed

The hyphen is not a reserved character in the URI spec https://tools.ietf.org/html/rfc3986#section-2.2
2018-02-11 23:49:18 +01:00
Eugen Rochko
e20700fe8f
Fix Chewy trying to update index with the wrong strategy (#6464) 2018-02-11 22:59:44 +01:00
Jenkins
5ffa6f0fa1 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-11 18:17:11 +00:00
Eugen Rochko
cf36d184f4
Interactive rake mastodon:setup task (#6451)
* Add better CLI prompt

* Add rake mastodon:setup interactive wizard

* Test db/redis/smtp configurations and add admin user at the end

* Test database connection even when database does not exist yet
2018-02-11 18:40:57 +01:00
Jenkins
efc9ffcb80 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-11 04:10:16 +00:00
David Yip
b6159bcb0a
Merge remote-tracking branch 'origin/merge-upstream' 2018-02-10 22:08:45 -06:00
beatrix
136458eaf4
Merge pull request #362 from ThibG/glitch-soc/fix-notifications-clearing
Revert 245816ab27 (fixes #348)
2018-02-10 21:47:27 -05:00
beatrix
20b519f0fc
Merge pull request #236 from glitch-soc/plaintext-mutes
Make keyword mutes operate on plain text input
2018-02-10 21:46:19 -05:00
David Yip
380989def8
Remove NOKOGIRI_USE_SYSTEM_LIBRARIES from Travis. #236.
Nokogiri linked to the version of libxml2 shipped in Travis' Trusty
image exhibits incorrect behavior when parsing strings resembling

    <p>I <3 oats</p>

In particular, Html2Text.convert (which uses nokogiri) will return "I"
for the above string when it ought to return "I <3 oats".

The version of libxml2 shipped with nokogiri 1.8.1+ exhibits correct
parse behavior for this string, as does the libxml2 present in Ubuntu
Xenial.
2018-02-10 19:29:17 -06:00
Thibaut Girka
fb6f131fa1 Revert 245816ab27 (fixes #348)
Since 245816ab27, IntersectionObserverArticle
assumes that its children do not change unless the number of children changes.

This is not the case with the notification overlay, which resulted in the
checkmark of notification cleaning mode not updating unless scrolling to make
notifications appear/disappear.

This change may negatively impact performances.
2018-02-10 20:59:50 +01:00
David Yip
f1f67c46c5
Use Html2Text to generate plaintext for keyword mutes. #236.
This allows us to match URLs inside link hrefs.
2018-02-10 10:36:16 -06:00
David Yip
9105b0c954
Introduce html2text for extracting plaintext from statuses. #236.
Unlike strip_tags, html2text will preserve text present in other nodes,
e.g. anchor tags:

    [1] pry(main)> str = '<a href="http://www.example.com">A link</a>'
    => "<a href=\"http://www.example.com\">A link</a>"
    [2] pry(main)> Html2Text.convert(str)
    => "[A link](http://www.example.com)"
    [3] pry(main)> include ActionView::Helpers::SanitizeHelper
    => Object
    [4] pry(main)> strip_tags(str)
    => "A link"

Preserving the href of an anchor allows keyword mutes to also match on
URLs, which is something that the frontend regex filter can currently
do.
2018-02-10 10:36:16 -06:00
David Yip
53c86b29f0
Glitch::FilterHelper -> Glitch::KeywordMuteHelper. #234.
The class helps out with keyword mutes, not just some general concept of
"filtering".
2018-02-10 10:36:15 -06:00
Kazushige Tominaga
718802a05d Added FetchRemoteAccountService spec (#6456)
* Added #link_header spec

* Added #call spec

* Delete spec of private methods

* Added #call spec
2018-02-10 17:10:57 +01:00
ThibG
411c9ecb4b Fix password recovery (#6459)
* Fix password recovery

* Use “resource” instead of “current_user”
2018-02-10 17:09:44 +01:00
David Yip
d263e3bc2d
Fill out some examples for Glitch::FilterHelper. #234.
Also add HTML entity decoding to Glitch::FilterHelper, which is needed
to e.g. match "<" to the tag-stripped version of "<p><3</p>" or
"<p>&lt;3</p>".
2018-02-10 02:40:27 -06:00
David Yip
29b5b46c87
Strip HTML from keyword mute input. #234. 2018-02-10 02:40:27 -06:00
Kazushige Tominaga
cbe8743e47 Added #call spec (#6455)
* Added #link_header spec

* Added #call spec

* Delete spec of private methods
2018-02-10 03:31:38 +01:00
Eugen Rochko
3ebc0ad4d3
Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00