Commit graph

1438 commits

Author SHA1 Message Date
Thibaut Girka
a84aedb7a7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
  Upstream raised max image size from 8MB to 10MB while our limit is
  configurable. Raised the default to 10MB.
2019-10-03 11:10:12 +02:00
Eugen Rochko
62f60e86c2
Fix account counters being overwritten by parallel writes (#12045) 2019-10-02 04:59:37 +02:00
ThibG
3a4d994c40 Fix BootstrapTimelineService crashing when bootstrapped accounts are invalid (#12037)
* Add test to handle suspended and missing users in BootstrapTimelineService

* Fix BootstrapTimelineService crashing when bootstrapped accounts are invalid
2019-10-01 15:10:00 +02:00
Eugen Rochko
9ba40a6bfd
Remove HEAD request from fetching link previews (#12028)
It is not really necessary and we need to reduce requests
2019-10-01 04:54:10 +02:00
Eugen Rochko
5c42f47617
Fix records not being indexed sometimes (#12024)
It's possible that after commit callbacks were not firing when
exceptions occurred in the process. Also, the default Sidekiq
strategy does not push indexing jobs immediately, which is not
necessary and could be part of the issue too.
2019-10-01 01:19:11 +02:00
Thibaut Girka
16ff7c5627 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- Gemfile.lock
- app/controllers/about_controller.rb
- app/controllers/auth/sessions_controller.rb
2019-09-30 12:23:57 +02:00
Eugen Rochko
5f69eb89e2
Add a nodeinfo endpoint (#12002)
* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* Add some additional node info stuff

* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* expanding this to include federation info

* codeclimate feedback

* CC feedback

* using activeserializers seems like a good idea...

* get rid of draft 2.1 version

* Reimplement 2.1, also fix metaData -> metadata

* Fix metaData -> metadata here too

* Fix nodeinfo 2.1 tests

* Implement cache for monthly user aggregate

* Useless

* Remove ostatus from the list of supported protocols

* Fix nodeinfo's open_registration reading obsolete setting variable

* Only serialize domain blocks with user-facing limitations

* Do not needlessly list noop severity in nodeinfo

* Only serialize domain blocks info in nodeinfo when they are set to be displayed to everyone

* Enable caching for nodeinfo endpoints

* Fix rendering nodeinfo

* CodeClimate fixes

* Please CodeClimate

* Change InstancePresenter#active_user_count_months for clarity

* Refactor NodeInfoSerializer#metadata

* Remove nodeinfo 2.1 support as the schema doesn't exist

* Clean-up
2019-09-29 21:31:51 +02:00
Eugen Rochko
ab33c4df94
Add exclude_unreviewed param to GET /api/v2/search REST API (#11977)
Make it so normal search returns even unreviewed matches, but
autosuggestions do not.

Fix #11960
2019-09-28 01:02:21 +02:00
ThibG
18b451c0e6 Change silences to always require approval on follow (#11975)
* Change silenced accounts to require approval on follow

* Also require approval for follows by people explicitly muted by target accounts

* Do not auto-accept silenced or muted accounts when switching from locked to unlocked

* Add `follow_requests_count` to verify_credentials

* Show “Follow requests” menu item if needed even if account is locked

* Add tests

* Correctly reflect that follow requests weren't auto-accepted when local account is silenced

* Accept follow requests from user-muted accounts to avoid leaking mutes
2019-09-27 21:13:51 +02:00
Eugen Rochko
3ed94dcc1a
Add account migration UI (#11846)
Fix #10736

- Change data export to be available for non-functional accounts
- Change non-functional accounts to include redirecting accounts
2019-09-19 20:58:19 +02:00
Thibaut Girka
5cadb47238 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/auth/sessions_controller.rb
  Minor conflict due to glitch-soc's theming code
2019-09-18 17:25:56 +02:00
Eugen Rochko
e1066cd431
Add password challenge to 2FA settings, e-mail notifications (#11878)
Fix #3961
2019-09-18 16:37:27 +02:00
Eugen Rochko
4f6af87906
Change spam check to apply to local accounts and add a threshold (#11806)
Instead of detecting spam on first duplicate message, add a
threshold of 5 such messages to reduce false positives
2019-09-18 12:53:13 +02:00
Eugen Rochko
a4b60e9ba4
Fix TOTP codes not being filtered from logs during enabling/disabling (#11877)
Not a serious issue because they are meaningless past single use
2019-09-18 02:48:40 +02:00
Eugen Rochko
c707ef49d9
Fix 2FA challenge and password challenge for non-database users (#11831)
* Fix 2FA challenge not appearing for non-database users

Fix #11685

* Fix account deletion not working when using external login

Fix #11691
2019-09-15 21:08:39 +02:00
Thibaut Girka
b83e2df6b5 Fix tests being broken because of different default setting 2019-09-13 20:28:22 +02:00
Thibaut Girka
74c5b2bd08 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
  Conflict because we changed the number of default results to be
  configurable
- app/lib/settings/scoped_settings.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
2019-09-13 18:13:43 +02:00
Eugen Rochko
18331fefa2
Remove deprecated GET /api/v1/search API (#11823)
Use `GET /api/v2/search` instead
2019-09-13 16:11:13 +02:00
Eugen Rochko
0762258aec
Fix hashtags being split by ZWNJ character (#11821)
Fix #11761
2019-09-13 16:01:26 +02:00
Eugen Rochko
c5d37f18cb
Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
2019-09-11 16:32:44 +02:00
Tao Bror Bojlén
4fe127664b add admin setting for default search engine indexing (fix #11750) (#11804) 2019-09-11 08:44:58 +02:00
ThibG
4faaa5b25e Add updated relationship to follow request API responses (#11800)
Fixes #11747
2019-09-10 20:56:42 +02:00
Eugen Rochko
1110ea1a91
Add batch actions and categories to admin UI for custom emojis (#11793) 2019-09-09 22:44:17 +02:00
Eugen Rochko
e445a8af64
Add timeline read markers API (#11762)
Fix #4093
2019-09-06 13:55:51 +02:00
Thibaut Girka
5088eb8388 Merge branch 'master' into glitch-soc/merge-upstream 2019-09-05 11:36:41 +02:00
ThibG
692c5b439a Fix ActivityPub context not being dynamically computed (#11746)
* Fix contexts not being dynamically included

Fixes #11649

* Refactor Note context in serializer

* Refactor Actor serializer
2019-09-03 22:52:32 +02:00
Eugen Rochko
70ddef2654
Change trending hashtags to not disappear instantly after midnight (#11712) 2019-09-02 18:11:13 +02:00
Thibaut Girka
9044a2b051 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/directories/index.html.haml
  Upstream has redesigned the profile directory, and we
  had a glitch-soc-specific change to hide follower counts.
  Ported that change to the new design.
2019-08-30 13:38:41 +02:00
Eugen Rochko
b54b725d6b
Fix uncaught domain normalization error in remote follow (#11703) 2019-08-30 02:19:17 +02:00
Eugen Rochko
22ce4778eb
Fix uncaught parameter missing exceptions and missing error templates (#11702) 2019-08-30 01:34:47 +02:00
Thibaut Girka
48b8a1f414 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- app/services/remove_status_service.rb
- db/schema.rb

All conflicts were due to the addition of a `deleted_at` attribute
to Statuses and reworked database indexes.
2019-08-29 12:07:50 +02:00
Eugen Rochko
73ca0bb925
Add option to include reported statuses in warning e-mail (#11639) 2019-08-23 22:37:23 +02:00
Eugen Rochko
97192d9a77
Fix remote and staff-removed statuses leaving media behind for a day (#11638)
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
2019-08-22 04:17:12 +02:00
Thibaut Girka
1488be7d96 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/home_controller.rb
- app/controllers/shares_controller.rb
- app/javascript/packs/public.js
- app/models/status.rb
- app/serializers/initial_state_serializer.rb
- app/views/home/index.html.haml
- app/views/layouts/public.html.haml
- app/views/public_timelines/show.html.haml
- app/views/shares/show.html.haml
- app/views/tags/show.html.haml
- config/initializers/content_security_policy.rb
- config/locales/en.yml
- config/webpack/shared.js
- package.json
2019-08-19 21:49:35 +02:00
Eugen Rochko
cc0a55cf9a
Add more accurate hashtag search (#11579)
* Add more accurate hashtag search

Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly

* Fix search analyzer dropping non-ascii characters
2019-08-18 03:45:51 +02:00
Eugen Rochko
e5cee8062f
Fix blurhash and autoplay not working on public pages (#11585) 2019-08-16 19:15:05 +02:00
Eugen Rochko
8fdff2748f
Add more accurate account search (#11537)
* Add more accurate account search

When ElasticSearch is available, a more accurate search is implemented:

- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)

The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.

* Support single-letter usernames

* Fix tests

* Fix not picking up account updates

* Add weights and normalization for scores, skip zero terms queries

* Use local counts for accounts index, adjust search parameters

* Fix mistakes

* Using updated_at of accounts is inadequate for remote accounts
2019-08-16 01:24:03 +02:00
Thibaut Girka
ebc0521ba8 Merge branch 'master' into glitch-soc/merge-upstream 2019-08-08 08:44:33 +02:00
ThibG
bced70469a Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
2019-08-07 20:20:23 +02:00
Thibaut Girka
04534604c6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- app/views/admin/dashboard/index.html.haml
- config/locales/simple_form.en.yml
2019-08-07 13:56:48 +02:00
Yusuke Nakamura
82d2069c75 Bump faker from 1.9.6 to 2.1.0 and update faker api (#11489)
* Bump faker from 1.9.6 to 2.1.0

Bumps [faker](https://github.com/stympy/faker) from 1.9.6 to 2.1.0.
- [Release notes](https://github.com/stympy/faker/releases)
- [Changelog](https://github.com/stympy/faker/blob/master/CHANGELOG.md)
- [Commits](https://github.com/stympy/faker/compare/1.9.6...v2.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Use faker api v2

https://github.com/stympy/faker/releases/tag/2.0
2019-08-06 15:33:03 +02:00
Thibaut Girka
cad2e6eb7a Merge branch 'master' into glitch-soc/merge-upstream 2019-08-06 13:16:53 +02:00
Eugen Rochko
115dab78f1
Change admin UI for hashtags and add back whitelisted trends (#11490)
Fix #271

Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.

When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.

The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.

The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
2019-08-05 19:54:29 +02:00
Thibaut Girka
e50554391a Merge branch 'master' into glitch-soc/merge-upstream 2019-08-05 13:13:28 +02:00
Eugen Rochko
8b9d0a0533
Remove XML version of Webfinger and remove links to Atom feeds (#11460)
Fix #11453
2019-08-01 19:14:02 +02:00
Eugen Rochko
92de439c04
Change hashtag search to only return results that have trended in the past (#11448)
* Change hashtag search to only return results that have trended in the past

A way to eliminate typos and other one-off "junk" results

* Fix excluding exact matches that don't have a score

* Fix tests
2019-07-30 20:29:50 +02:00
ThibG
ff789a751a Fix boosting & unboosting preventing a boost from appearing in the TL (#11405)
* Fix boosting & unboosting preventing a boost from appearing in the TL

* Add tests

* Avoids side effects when aggregate_reblogs isn't true
2019-07-30 13:18:23 +02:00
Thibaut Girka
24968d20a0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/directories_controller.rb
- package.json
- yarn.lock
2019-07-30 12:22:33 +02:00
Eugen Rochko
24552b5160
Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
Eugen Rochko
e136112ab7
Fix tag normalization and migration not removing duplicate tags (#11441)
Fix #11428
2019-07-29 20:40:21 +02:00
Eugen Rochko
b9b0313c78
Revert "Remove conversation URI (#11423)" (#11424)
This reverts commit 75f7f9930e.
2019-07-28 17:47:37 +02:00
Eugen Rochko
75f7f9930e
Remove conversation URI (#11423)
It is not part of ActivityPub and will free up a lot of space
2019-07-28 17:30:12 +02:00
Thibaut Girka
bca3825c17 Merge branch 'master' into glitch-soc/merge-upstream 2019-07-28 16:28:05 +02:00
ysksn
d6ada2eb30 Implement pending tests (#11415) 2019-07-27 10:24:26 +02:00
Eugen Rochko
b9fbcbfe4e
Add search syntax for operators and phrases (#11411) 2019-07-27 04:42:08 +02:00
ThibG
92569ffde8 Fix invites not being disabled upon account suspension (#11412)
* Disable invite links from disabled/suspended users

* Add has_many invites relationship to users

* Destroy unused invites when suspending an account
2019-07-26 18:55:33 +02:00
Thibaut Girka
2d35e4460a Merge branch 'master' into glitch-soc/merge-upstream 2019-07-24 16:23:53 +02:00
Rey Tucker
94f5c714f1 Don't delete periods when validating username uniqueness (#11392) (#11400)
* Check to make sure usernames with '.' cannot be created

* Add test for instance actor account name conflicts

This makes sure that migration 20190715164535_add_instance_actor
won't fail if there's already an account that is named the same
as the domain (minus the .)

* Put the test into the correct context...

* Add another test to split this into two validations

* Don't delete periods when validating username uniqueness (#11392)

The 20190715164535_add_instance_actor migration fails if there's
already a username similar to the domain name, e.g. if you are
'vulpine.club' and have a user named 'vulpineclub', validation
fails.

Upon further review, usernames with periods are dropped by the
regular expression in the Account class, so we don't need to
worry about it here.

Fixes #11392
2019-07-24 14:19:17 +02:00
Thibaut Girka
444796b69b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/application_controller.rb
- app/controllers/auth/confirmations_controller.rb
- app/controllers/auth/sessions_controller.rb
- app/controllers/settings/deletes_controller.rb
- app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
2019-07-23 10:51:07 +02:00
Eugen Rochko
964ae8eee5
Change unconfirmed user login behaviour (#11375)
Allow access to account settings, 2FA, authorized applications, and
account deletions to unconfirmed and pending users, as well as
users who had their accounts disabled. Suspended users cannot update
their e-mail or password or delete their account.

Display account status on account settings page, for example, when
an account is frozen, limited, unconfirmed or pending review.

After sign up, login users straight away and show a simple page that
tells them the status of their account with links to account settings
and logout, to reduce onboarding friction and allow users to correct
wrongly typed e-mail addresses.

Move the final sign-up step of SSO integrations to be the same
as above to reduce code duplication.
2019-07-22 10:48:50 +02:00
ThibG
7de8c51873 Play animated custom emoji on hover (#11348)
* Play animated custom emoji on hover in status

* Play animated custom emoji on hover in display names

* Play animated custom emoji on hover in bios/bio fields

* Add support for animation on hover on public pages emojis too

* Fix tests

* Code style cleanup
2019-07-21 18:10:40 +02:00
Eugen Rochko
bd1545de5e
Change locale detection to run once per session (#8657)
Fix #6462
2019-07-21 18:08:02 +02:00
Eugen Rochko
bd87e66679
Remove WebSub subscriptions (#11303) 2019-07-21 04:08:00 +02:00
Thibaut Girka
055450bc02 Merge branch 'master' into glitch-soc/merge-upstream 2019-07-19 23:37:41 +02:00
ThibG
c37c1da41e Disallow numeric-only hashtags (#11363)
* Add spec covering numeric-only hashtags

* Fix hashtag regex
2019-07-19 23:22:35 +02:00
Thibaut Girka
3407ae8683 Fix sanitizer text case for glitch-soc, which preserves lists 2019-07-19 19:16:07 +02:00
Thibaut Girka
249991c498 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile.lock
- app/controllers/accounts_controller.rb
- app/controllers/admin/dashboard_controller.rb
- app/controllers/follower_accounts_controller.rb
- app/controllers/following_accounts_controller.rb
- app/controllers/remote_follow_controller.rb
- app/controllers/stream_entries_controller.rb
- app/controllers/tags_controller.rb
- app/javascript/packs/public.js
- app/lib/sanitize_config.rb
- app/models/account.rb
- app/models/form/admin_settings.rb
- app/models/media_attachment.rb
- app/models/stream_entry.rb
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- app/services/batched_remove_status_service.rb
- app/services/post_status_service.rb
- app/services/process_mentions_service.rb
- app/services/reblog_service.rb
- app/services/remove_status_service.rb
- app/views/admin/settings/edit.html.haml
- config/locales/simple_form.pl.yml
- config/settings.yml
- docker-compose.yml
2019-07-19 18:26:49 +02:00
ThibG
fda437a020 Fix sanitizing lists contents (#11354)
* Add test

* Fix code for sanitizing nested lists stripping all tags
2019-07-19 01:44:58 +02:00
ThibG
730c4053d6 Add ActivityPub actor representing the entire server (#11321)
* Add support for an instance actor

* Skip username validation for local Application accounts

* Add migration script to create instance actor

* Make Codeclimate happy

* Switch to id -99 for instance actor

* Remove unused `icon` and `image` attributes from instance actor

* Use if/elsif/else instead of return + ternary operator

* Add instance actor to fresh installs

* Use instance actor as instance representative

Use instance actor for forwarding reports, relay operations, and spam
auto-reporting.

* Seed database in test environment

* Fix single-user mode

* Fix tests

* Fix specs to accomodate for an extra `Account`

* Auto-reject follows on instance actor

Following an instance actor might make sense, but we are not handling that
right now, so auto-reject.

* Fix webfinger lookup and serialization for instance actor

* Rename instance actor

* Make it clear in the HTML view that the instance actor should not be blocked

* Raise cache time for instance actor as there's no dynamic content

* Re-use /about/more with a flash message for instance actor profile
2019-07-19 01:44:42 +02:00
Eugen Rochko
84e988479e
Fix only one middle dot being recognized in hashtags (#11345)
Fix #10934
2019-07-18 03:02:56 +02:00
Eugen Rochko
5bfe1e1f05
Change language detection to include hashtags as words (#11341) 2019-07-18 03:02:15 +02:00
Eugen Rochko
6ff67be0f6
Add a spam check (#11217)
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
2019-07-13 16:45:50 +02:00
Eugen Rochko
5bf67ca913
Add ActivityPub secure mode (#11269)
* Add HTTP signature requirement for served ActivityPub resources

* Change `SECURE_MODE` to `AUTHORIZED_FETCH`

* Add 'Signature' to 'Vary' header and improve code style

* Improve code style by adding `public_fetch_mode?` method
2019-07-11 20:11:09 +02:00
Eugen Rochko
4e8dcc5dbb
Add HTTP signatures to all outgoing ActivityPub GET requests (#11284) 2019-07-11 14:49:55 +02:00
Eugen Rochko
5d3feed191
Refactor fetching of remote resources (#11251) 2019-07-10 18:59:28 +02:00
Eugen Rochko
4e92183227
Refactor domain block checks (#11268) 2019-07-09 03:27:35 +02:00
Eugen Rochko
ef15246397
Remove unused remote unfollow controller (#11250) 2019-07-08 12:04:06 +02:00
Eugen Rochko
63c7fe8e48
Refactor controllers for statuses, accounts, and more (#11249) 2019-07-08 12:03:45 +02:00
Eugen Rochko
b851456139
Remove Atom feeds and old URLs in the form of GET /:username/updates/:id (#11247) 2019-07-07 16:16:51 +02:00
Thibaut Girka
7039dca12c Merge commit 'c07cca4727041ea5a5721acbc603d4bfb45a15a6' into glitch-soc/merge-upstream
Unlike upstream, kept the direct timeline endpoint, as it is still of use in
glitch-soc.
2019-07-07 13:49:31 +02:00
Eugen Rochko
23aeef52cc
Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
Eugen Rochko
0c1b1069c9
Remove deprecated REST API GET /api/v1/statuses/:id/card (#11213) 2019-07-05 02:15:24 +02:00
Eugen Rochko
3fd6ab99e6
Remove deprecated REST API GET /api/v1/timelines/direct (#11212) 2019-07-05 02:14:56 +02:00
Thibaut Girka
6ab84c12a7 Merge branch 'master' into glitch-soc/merge-upstream 2019-07-04 16:21:39 +02:00
Eugen Rochko
0d9ffe56fb
Add request pool to improve delivery performance (#10353)
* Add request pool to improve delivery performance

Fix #7909

* Ensure connection is closed when exception interrupts execution

* Remove Timeout#timeout from socket connection

* Fix infinite retrial loop on HTTP::ConnectionError

* Close sockets on failure, reduce idle time to 90 seconds

* Add MAX_REQUEST_POOL_SIZE option to limit concurrent connections to the same server

* Use a shared pool size, 512 by default, to stay below open file limit

* Add some tests

* Add more tests

* Reduce MAX_IDLE_TIME from 90 to 30 seconds, reap every 30 seconds

* Use a shared pool that returns preferred connection but re-purposes other ones when needed

* Fix wrong connection being returned on subsequent calls within the same thread

* Reduce mutex calls on flushes from 2 to 1 and add test for reaping
2019-07-02 00:34:38 +02:00
Thibaut Girka
3922b518f7 Merge branch 'master' into glitch-soc/merge-upstream 2019-06-28 21:36:50 +02:00
Eugen Rochko
e64e6a03dd
Add categories for custom emojis (#11196)
Fix #7940
2019-06-28 15:54:10 +02:00
Thibaut Girka
c8ba75b963 Merge branch 'master' into glitch-soc/merge-upstream 2019-06-28 12:11:45 +02:00
ThibG
9a90ec3b3b Fix account URI in UpdatePollSerializer (#11194)
* Fix account URI in UpdatePollSerializer

Fixes #11185

* Add specs
2019-06-27 19:41:55 +02:00
Thibaut Girka
aaec64a500 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/preferences_controller.rb
- app/lib/user_settings_decorator.rb
- app/models/user.rb
- config/locales/simple_form.en.yml
2019-06-26 23:19:22 +02:00
ThibG
915c619394 Add support for Audio activities (#11189)
Fixes #11127
2019-06-26 19:32:36 +02:00
ThibG
47ef4a6c7a Apply filters to poll options (#11174)
* Apply filters to poll options in WebUI

Fixes #11128

* Apply filters to poll options server-side

* Add poll options to searchable text
2019-06-25 14:45:14 +02:00
Thibaut Girka
ddd875ad99 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
  Upstream added audio attachment support
- app/serializers/initial_state_serializer.rb
  Upstream added audio attachment support and how mimetypes are returned
- app/serializers/rest/instance_serializer.rb
  Upstream added a few fields
- config/application.rb
  Upstream added a different paperclip transcoder
2019-06-24 15:02:59 +02:00
Eugen Rochko
707ddf7808
Change domain blocks to automatically support subdomains (#11138)
* Change domain blocks to automatically support subdomains

If a more authoritative domain is blocked (example.com), then the
same block will be applied to a subdomain (foo.example.com)

* Match subdomains of existing accounts when blocking/unblocking domains

* Improve code style
2019-06-22 00:13:10 +02:00
Eugen Rochko
7696f77245
Add moderation API (#9387)
Fix #8580
Fix #7143
2019-06-20 02:52:34 +02:00
Thibaut Girka
356e9150df Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/lib/sanitize_config.rb
  Keep our version, we support the tags upstream transforms.
- package.json
- yarn.lock
2019-06-19 18:36:16 +02:00
Eugen Rochko
103a9f4466
Fix sanitizer making block level elements unreadable (#10836)
Fix #10834
2019-06-16 21:46:36 +02:00
Thibaut Girka
1b0ff4cd69 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/notifications_controller.rb
- app/javascript/packs/public.js
- app/views/settings/preferences/show.html.haml
- app/views/stream_entries/_simple_status.html.haml
- config/locales/simple_form.en.yml
- config/locales/simple_form.pl.yml
- config/navigation.rb
- config/routes.rb
2019-06-10 18:59:53 +02:00
Thibaut Girka
34b8346e7f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/statuses_controller.rb
- app/controllers/stream_entries_controller.rb
2019-06-07 17:00:36 +02:00
Eugen Rochko
560ec24e58
Change /settings/preferences to redirect to appearance, add /settings/preferences/other (#10988) 2019-06-07 16:51:08 +02:00
Eugen Rochko
1db4117030
Change preferences page into appearance, notifications, and other (#10977) 2019-06-07 03:39:24 +02:00
ThibG
6c464cd424 Do not misattribute inlined boosts if attributedTo isn't present (#10967)
* Do not misattribute inlined boosts if `attributedTo` isn't present

Fixes #10950

* Fix tests
2019-06-04 23:24:31 +02:00
Thibaut Girka
02b56c7e1a Merge remote-tracking branch 'upstream/master' into glitch-soc/merge-upstream 2019-06-04 20:40:19 +02:00
Eugen Rochko
48fee1a800
Fix poll API not requiring authentication on non-public polls (#10960)
* Fix poll API not requiring authentication on non-public polls

That API does not reveal the content of the status, i.e. the question
itself, nor who the author is, nor which status it belongs to, but it
does reveal the poll options and how many answers they got

Fix #10959

* Add test
2019-06-04 20:10:26 +02:00
Thibaut Girka
76918554a6 Merge remote-tracking branch 'upstream/master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile.lock
- config/webpack/rules/css.js
- package.json
- yarn.lock
2019-06-03 14:54:30 +02:00
ThibG
a353dc6030 Fix NotifyService test with regards to reblogs (#10928)
Fixes #10890
2019-06-02 18:08:26 +02:00
Thibaut Girka
c0dc247bce Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/account.rb
- app/views/settings/profiles/show.html.haml
- spec/controllers/api/v1/accounts/credentials_controller_spec.rb

Conflicts were due to an increase in account bio length upstream, which
is already covered in glitch-soc through `MAX_BIO_CHARS`.
2019-05-23 19:01:30 +02:00
trwnh
e3b39ea4a4 Update remote bio test from 160 to 500 (#10799) 2019-05-21 13:29:06 +02:00
Paul Woolcock
0c933c1b8c Add account_id param to GET /api/v1/notifications (#10796)
* Add `from_account` to notifications API

this adds the ability to filter notifications by the account they
originated from

* passing a non-existent user should cause none to be returned

* Fix codeclimate warnings

* fix more codeclimate warnings

* make requested changes:

* use account id instead of user@domain
* name the param `account_id` instead of `from_account`

* Don't use `return` in a lambda
2019-05-21 13:28:49 +02:00
trwnh
a6caf919e2 Change bio limit from 160 to 500 (#10790)
* Change note_length validator from 160 to 500

* Change input maxlength from 160 to 500

* update bio test from 160 to 500

* Multiply a string 30 times instead of 10
2019-05-19 22:51:44 +02:00
Thibaut Girka
08a2af27c3 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-19 21:53:44 +02:00
ThibG
ae18386558 Fix “invited by” not showing up for invited accounts in admin interface (#10791) 2019-05-19 21:40:36 +02:00
Thibaut Girka
6e227ecb04 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-18 10:04:32 +02:00
ThibG
a1519a8ef5 Prevent from publicly boosting one's own private toots (#10775) 2019-05-18 00:28:51 +02:00
Thibaut Girka
6badf2d252 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-15 17:11:40 +02:00
ThibG
14f6ce2885 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
Thibaut Girka
68629f2773 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/statuses_controller.rb
  minor conflict because of glitch-soc's theming system
- app/controllers/stream_entries_controller.rb
  minor conflict because of glitch-soc's theming system
2019-05-10 17:09:12 +02:00
ThibG
62f5235b6f Prevent silenced local users from notifying remote users not following them (#10575)
* Prevent silenced local users from notifying remote users not following them

This is an attempt to extend the local restrictions of silenced users to the
federation.

* Add tests

* Add tests for making sure private status don't get sent over OStatus
2019-05-09 22:05:43 +02:00
Thibaut Girka
33c80e0783 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
2019-05-04 16:37:26 +02:00
Eugen Rochko
7cb369d4c6
Change e-mail whitelist/blacklist to not be checked when invited (#10683)
* Change e-mail whitelist/blacklist to not be checked when invited

And only when creating an account, not when updating it later

Fix #10648

* Fix test
2019-05-03 23:44:44 +02:00
ThibG
011b032300 Provide a link to existing domain block when trying to block an already-blocked domain (#10663)
* When trying to block an already-blocked domain, provide a link to the block

* Fix styling for links in flash messages

* Allow blocks to be upgraded but not downgraded
2019-05-03 20:36:36 +02:00
ThibG
21a73c52a7 Check that an invite link is valid before bypassing approval mode (#10657)
* Check that an invite link is valid before bypassing approval mode

Fixes #10656

* Add tests

* Only consider valid invite links in registration controller

* fixup
2019-05-02 04:30:12 +02:00
Thibaut Girka
2f422b708f Merge branch 'master' into glitch-soc/merge-upstream 2019-04-10 21:19:21 +02:00
Eugen Rochko
a9f130b8d8
Fix Keybase verification using wrong domain for remote accounts (#10547) 2019-04-10 20:28:43 +02:00
Alex Gessner
154106c0c3 compare usernames case-insensitively on new proof creation flow (#10544)
* compare usernames case-insensitively on new proof creation flow

* Fix code style issue
2019-04-10 18:05:11 +02:00
Thibaut Girka
5085559b15 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/admin/pending_accounts/index.html.haml
  No real conflict, upstream changes too close to glitch-specific
  theming system changes.
- config/navigation.rb
  Upstream redesigned the settings navigation, took those changes
  and re-inserted the flavours menu.
2019-04-10 09:16:08 +02:00
Eugen Rochko
46cb36fd2c
Add invite request to pending account notification e-mail (#10528)
Fix sorting of the pending accounts page
2019-04-10 00:36:01 +02:00
Hinaloe
48f466daf1 Allow set the voting period to just 5 minutes (#10525)
* Add spec of PollValidator for #10190

* Raise fraction less than 1 second

* format

* simplify time initialize
2019-04-09 17:02:12 +02:00
Eugen Rochko
8b69a66380 Add "why do you want to join" field to invite requests (#10524)
* Add "why do you want to join" field to invite requests

Fix #10512

* Remove unused translations

* Fix broken registrations when no invite request text is submitted
2019-04-09 23:06:30 +09:00
Thibaut Girka
f5f6d23d55 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/pl.yml
  Conflict caused by new upstream string too close to glitch-specific
  “flavour” string. Took both strings.
2019-04-08 15:57:56 +02:00
ThibG
cb71c95e22 Export and import show_reblogs together with following list (#10495)
* Refactor imports

* Export show_reblogs when exporting list of followed users

* Add support for importing show_reblogs with following collection

* Fix tests
2019-04-08 07:28:27 +02:00
Eugen Rochko
67b3b62b98
Improve blocked view of profiles (#10491)
* Revert "Fix filtering of favourited_by, reblogged_by, followers and following (#10447)"

This reverts commit 120544067f.

* Revert "Hide blocking accounts from blocked users (#10442)"

This reverts commit 62bafa20a1.

* Improve blocked view of profiles

- Change "You are blocked" to "Profile unavailable"
- Hide following/followers in API when blocked
- Disable follow button and show "Profile unavailable" on public profile as well
2019-04-07 04:59:13 +02:00
Thibaut Girka
1682ac5717 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/following_accounts_controller.rb
  Conflicts were due to glitch-soc's theming system.
- app/javascript/packs/public.js
  Some code has been change upstream, but it has been
  moved to app/javascript/core/settings.js in glitch-soc.
  Applied the changes there.
2019-04-04 14:53:52 +02:00
ThibG
d4882aa64a Export and import hide_notifications alongside user mutes (#10335)
* Export hide_notifications along with user mutes

* Import hide_notifications along with muted users list

* Add headers for CSV exports
2019-04-03 18:17:43 +02:00
Thibaut Girka
12dae9d583 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/form/admin_settings.rb
- config/locales/ja.yml
2019-04-01 21:28:31 +02:00
ThibG
62bafa20a1 Hide blocking accounts from blocked users (#10442)
* Revert "Add indication that you have been blocked in web UI (#10420)"

This reverts commit bd02ec6daa.

* Revert "Add `blocked_by` relationship to the REST API (#10373)"

This reverts commit 9745de883b.

* Hide blocking accounts from search results

* Filter blocking accouts from account followers

* Filter blocking accouts from account's following accounts

* Filter blocking accounts from “reblogged by” and “favourited by” lists

* Remove blocking account from URL search

* Return 410 on trying to fetch user data from a user who blocked us

* Return 410 in /api/v1/account/statuses for suspended or blocking accounts

* Fix status filtering when performing URL search

* Restore some React improvements

Restore some cleanup from bd02ec6daa

* Refactor by adding `without_blocking` scope
2019-04-01 20:06:13 +02:00
ThibG
2acd8940de Fix more keybase-related test failures (#10443) 2019-04-01 19:10:44 +02:00
slice
85973f4f37 Improvements to image upload validation and creation (#10431)
* Check if image value is nil? before creating an image

Check if uploaded images aren't nil before creating SiteUpload models
for them.

* Validate presence of file in SiteUpload

* Fix file presence validation

* Fabricate SiteUpload#file

* Add link to Creative Commons license
2019-04-01 07:30:46 +02:00
Thibaut Girka
b72950fbd3 Fix failing keybase-related test 2019-03-30 18:19:40 +01:00
ThibG
abecaba317 Fix failing keybase-related test (#10428) 2019-03-30 18:15:23 +01:00
Thibaut Girka
4444d8f9b3 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-30 12:36:24 +01:00
Eugen Rochko
1714ea5978
Add ActivityPub representation for identity proofs (#10414)
* Add ActivityPub representation for identity proofs

* Add tests
2019-03-30 02:12:06 +01:00
Thibaut Girka
ce7d055d3c Merge branch 'master' into glitch-soc/merge-upstream 2019-03-28 18:35:25 +01:00
Alex Gessner
69141dca26 squashed identity proof updates (#10375) 2019-03-28 18:01:09 +01:00
Thibaut Girka
050efbc126 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/workers/activitypub/distribute_poll_update_worker.rb
- config/locales/pl.yml
2019-03-28 13:01:33 +01:00
Eugen Rochko
f1bc90ab50
Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401)
Also, fix some n+1 queries

Resolve #10365
2019-03-28 04:44:59 +01:00
Eugen Rochko
11fe293e1b
Remove unused ActivityPub @context values depending on response (#10378)
Fix #8078
2019-03-27 15:55:23 +01:00
Thibaut Girka
fcce135d27 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/admin/settings_controller.rb
- app/models/form/admin_settings.rb

Conflicts caused by upstream refactoring, while we have
flavours and skins, with the flavour_and_skin pseudo-setting.
2019-03-26 16:01:59 +01:00
Eugen Rochko
555c4e11ba
Add validations to admin settings (#10348)
* Add validations to admin settings

- Validate correct HTML markup
- Validate presence of contact username & e-mail
- Validate that all usernames are valid
- Validate that enums have expected values

* Fix code style issue

* Fix tests
2019-03-23 14:07:04 +01:00
Thibaut Girka
cbf1d711ba Merge branch 'master' into glitch-soc/merge-upstream 2019-03-20 17:32:39 +01:00