Commit Graph

209 Commits

Author SHA1 Message Date
Christian Pauly e6182db16b refactor: Remove deprecated fluffybox 2022-10-28 12:56:20 +02:00
Nicolas Werner c6a8f5a162 feat: Implement a queue for events pending decryption
fixes https://gitlab.com/famedly/company/frontend/famedly-web/-/issues/535
2022-10-12 11:07:50 +02:00
henri2h 7b61f8eb01 fix: don't assume redacts attribute from content to be valid 2022-09-12 17:09:29 +02:00
Nicolas Werner 43ab43966f fix: properly handle events not already in the db
fixes https://gitlab.com/famedly/app/-/issues/1798
2022-09-09 17:16:58 +02:00
Nicolas Werner 6555f36d6d feat: switch from pedantic to lints 2022-08-15 14:15:51 +00:00
Lanna Michalke 8058f88fe2 fix: missing null check
Signed-off-by: Lanna Michalke <l.michalke@famedly.com>
2022-08-01 08:16:35 +02:00
Lanna Michalke 1cb04ef9e4 fix: make Hive Collection path nullable
Signed-off-by: Lanna Michalke <l.michalke@famedly.com>
2022-08-01 07:19:05 +02:00
Christian Pauly 7885281c72 refactor: Use import sorter and ci templates 2022-07-21 15:44:04 +02:00
Christian Pauly 95b060a299 fix: Database did not get cleared correctly 2022-07-08 12:46:17 +02:00
TheOneWithTheBraid 3f0a177abf fix: BoxCollection not re-assignable
- make BoxCollection field in Hive collection DB re-assignable
- used by session import

Signed-off-by: TheOneWithTheBraid <the-one@with-the-braid.cf>
2022-06-10 06:17:29 +02:00
Lanna Michalke 039062bfdf feat: implement session export
- implement session export
- implement session import

Please note, this is not device dehydration but a fully offline session
dump eligible for e.g. TOR users.

Signed-off-by: Lanna Michalke <l.michalke@famedly.com>
2022-05-23 12:06:31 +02:00
Lanna Michalke 8fd3bbe28c feat: support HiveCollections as Database provider
- support Hive's new `CollectionBox` and `BoxCollection`

Signed-off-by: Lanna Michalke <l.michalke@famedly.com>
2022-05-21 20:19:52 +02:00
Henri Carnot bfdd4c810e Henri/fragmented timeline v2 2022-05-16 10:54:42 +00:00
Krille Fear 9a87850092 Revert "feat: non cached fragmented timeline"
This reverts commit 09b324674e
2022-05-12 09:46:29 +02:00
Christian Pauly b8ea213f73 fix: Missing null check in get single room method 2022-05-11 09:45:01 +02:00
Henri Carnot 09b324674e feat: non cached fragmented timeline 2022-05-03 14:08:33 +00:00
Christian Pauly 5973e629fe feat: Get event from push notification 2022-04-14 17:01:05 +02:00
Henri Carnot 809fe9e621 fix: room members loading
States were used before being fetched from the database.
Thus, room membership states weren't set, and so,
user display names weren't be fetched from the database.
2022-03-24 08:04:17 +00:00
Krille Fear 3bce8b4489 refactor: Remove unnecessary type checks and imports 2021-12-23 11:46:45 +01:00
Krille Fear e5fde4f801 chore: bump version 2021-12-16 08:17:36 +00:00
Krille Fear ac06864627 feat: Migrate olm sessions on database migration
This adds a getAllOlmSessions
endpoint to the database API and
implements them in both
implementations. This also
adds it to the database
migration.
2021-11-26 08:17:43 +01:00
Krille Fear 01eb851364 fix: 2021-11-23 15:48:18 +01:00
Krille Fear b99a78476a refactor: Remove Sembast database implementation
It was a nice experiment but it
loading everything into memory
is just too slow for big
accounts.
2021-11-18 19:55:21 +01:00
Krille Fear f5051a5afe feat: Implement sembast store
refactor: Use typed store
2021-11-16 09:50:25 +01:00
Krille Fear b7565af56f fix: use originServerTs to check if state event is old
Due to server bugs or whatever it sometimes
happens that old state events appear
in the setState method in the room class.
Previously we checked if we already know
this event ID, but for this we needed to
check the timeline which is very fluid.
Also this is a database operation in a
non-async method which works in Hive but
not in Sembast.
Using originServerTs is not 100% safe as
well but should be more stable because
the chance that servers have veeery wrong
time (which is necessary here) is much
lower than the risk that the timeline
is not long enough to know the
old event.
2021-11-15 10:57:55 +01:00
Krille Fear 872b3bff94 fix: Wrong null check in hive database 2021-11-10 09:09:24 +01:00
Krille Fear 315ed7bdcb feat: Add more benchmarks for sync, timeline, init
This also disables benchmarks if loglevel is higher than debug.
2021-11-09 10:03:59 +01:00
Krille Fear 9be87918f1 refactor: Make room in Event class not nullable 2021-11-04 14:21:04 +01:00
Krille Fear 766b27427a Merge branch 'td/invite-update' into 'main'
fix: add room invite update to roomStateBox, so invites don't show empty room when app is restarted

Closes #228

See merge request famedly/company/frontend/famedlysdk!865
2021-10-29 13:17:30 +00:00
Nicolas Werner 6e20c53b01
refactor: remove redundant null checks 2021-10-28 14:32:09 +02:00
Nicolas Werner ebd5bc3d24
refactor: make room nullsafe 2021-10-28 00:54:25 +02:00
Jayesh Nirve b955318edb
fix: add room invite update to roomStateBox, so invites don't show empty room when app is restarted 2021-10-26 16:00:22 +05:30
Nicolas Werner fb0a84d7b2
refactor: make client nullsafe 2021-10-25 16:34:02 +02:00
Krille Fear 7d5652e9a3 fix: Remove duplicated import 2021-10-25 11:11:17 +02:00
Jindra 2b8782c699 fix: delete forgotten files caused by mergetool 2021-10-14 15:37:43 +02:00
Jindřich Pikora c6e0359522 refactor: Change event status to enum 2021-10-14 13:31:07 +00:00
Nicolas Werner 60956bde00 chore: port the remaining encryption files to nullsafety 2021-10-13 13:58:21 +00:00
Krille Fear 3603dae312 feat: Calc benchmarks for hive operations on init 2021-10-07 12:18:13 +00:00
Christian Pauly 687a6341f1 fix: Sent events are sorted in SENDING timeline
Events with a status of 1 should be sorted in the normal timeline.
They should not be stucked at the bottom. This fixes a bug
where a limited timeline flag
can stuck a SENT event at the bottom of
the chat forever.
2021-10-04 17:45:27 +00:00
Nicolas Werner a196b53219 refactor: nullsafe encryption 2021-09-30 12:58:37 +00:00
Sorunome 259c9cade6
fix: missing range check
When requesting history the `start` parameter could become larger than the loaded events
from the database were, resulting in an error when attempting to request history.
2021-09-30 14:34:28 +02:00
Lukas Lihotzki 1d0202e14e refactor: null safe matrix_id_string_extension 2021-09-27 12:59:51 +00:00
Jayesh Nirve 77ca7523d0 refactor: remove unused clientId 2021-09-24 16:05:05 +02:00
Sorunome 80be9629bf
fix: Setting stuff in the database assumed event relationships were well-formatted 2021-09-21 11:31:53 +02:00
Krille Fear dec32975e2 fix: Clearing corrupted boxes
If a box is corrupted the clear function fails on it. Then
we should delete the box from the disk.
Currently we use the Hive.deletefromDisk() method which does not
work because it deletes only open boxes, but the box is obviously not open in
this case.
2021-09-14 10:33:11 +02:00
Krille Fear 00cc439122 fix: Do not set old events as state events
Previously we had a check which uses the old
sortOrder value.
This check has been removed with the refactoring which leads to
bug #209. This fixes it by checking if the
event is already known in the database.
I am not 100% happy with this solution as this database api is impossible
to be implemented with a sqlite db. Once we start to refactor the whole sync update logic
we maybe could find a better way, but only the fox god knows.
2021-09-14 08:29:54 +02:00
Christian Pauly 00771fc209 refactor: _updateUserDeviceKeys method 2021-09-13 14:58:07 +02:00
Christian Pauly e13b00d127 refactor: Make RoomUpdate class null safe by removing it
RoomUpdate came from a time where we had no data model for
SyncUpdates but now we have and therefore this class is just
code duplication. This removes the class
and uses the SyncRoomUpdate class from
the package matrix_api_lite instead.
It needed a lot of refactoring at some places
where I also have removed some unnecessary null or type checks.
2021-09-08 12:31:55 +00:00
Christian Pauly 210c925a0a fix: Do not handle sending event updates which are already synced
This is an edge case which might occour
on unstable data connections. The user sends an event and receives the sync
before the response to the sending
http request. This leads to duplicated
events while the response actually
should be ignored at this point.
2021-09-08 11:33:36 +02:00
Christian Pauly 524b09c572 feat: Do not load all timeline events from store at once 2021-09-03 14:58:43 +02:00
Christian Pauly 9c1f79359e refactor: Replace all logic regarding sortOrder
The current implementation of sortOrder can be made way more easier now
by just keeping the sortOrder of the list
and the timelineFragments in the hiveStore. This needed a huge
change but mostly removes a lot of code which can be done
way more easy now. This also needed some rewriting of the setState logic and changes to
the prevEvent calculation. This solution should also be more stable.
More information:
https://www.reddit.com/r/fluffychat/comments/pfnlhq/the_sort_order_of_matrix_timelines/
2021-09-02 07:13:38 +00:00
Sorunome efb5842959
fix: Implement dummy transactions for hive
That way some concurrency bugs might be fixed, such as if two sync
requests are processed at the same time. That can e.g. happen if you
request history while a sync request is already being processed.
2021-09-01 19:11:46 +02:00
Nicolas Werner 8b46fa3fc2 fix: room account data key/type returned encoded
We uri encode all hive keys. But in some cases we never decode them.
This leads to fun issues like SSSS being unreadable after restart.

fixes https://gitlab.com/famedly/company/frontend/famedlysdk/-/issues/179
2021-09-01 16:37:51 +00:00
Christian Pauly 62694248b9 refactor: Make UIA Request null safe 2021-08-30 09:26:34 +02:00
Christian Pauly 2589206851 refactor: Null safe QueuedToDeviceEvent and fix CamelCase filename 2021-08-30 09:20:17 +02:00
Christian Pauly 1fd40dd186 refactor: Make database null safe 2021-08-30 09:20:17 +02:00
Christian Pauly d75b27a0cd refactor: Ignore import_of_legacy_library_into_null_safe 2021-08-30 09:20:17 +02:00
Jayesh Nirve 42614bdfbd
Merge branch 'main' of gitlab.com:famedly/company/frontend/famedlysdk into td/fix-lastevent-preview 2021-08-26 02:42:36 +05:30
Lukas Lihotzki 9087f08775 chore: mxc url in Uri 2021-08-23 11:05:47 +02:00
Jayesh Nirve 164a95ea29 clean stuff up? 2021-08-23 07:34:57 +00:00
Jayesh Nirve c4c1c676da fix null check 2021-08-23 07:34:57 +00:00
Jayesh Nirve 572402980c fix null check 2021-08-23 07:34:57 +00:00
Jayesh Nirve 38235415eb fix: lastEvent in room preview now crespects edits order 2021-08-23 07:34:57 +00:00
Christian Pauly 9b7474e2f2 refactor: Update to dart 2.12 and opt out in all files
This also adds some missing license headers, I have detected while
using license header search&replace to add the
null safety opt out
2021-08-18 08:09:39 +02:00
Christian Pauly 86041513f8 refactor: Remove deprecated moor database
We have used some data models which were only used in moor in the tests.
I needed to rewrite them in the original data as well.
Also now the "fake database" on native is the same like on web now with hive.
2021-08-17 10:11:59 +02:00
Marcus f91f662c75 chore: update to matrix_sdk_lite 0.4.1 2021-08-05 12:10:45 +00:00
Lukas Lihotzki 46b2f8898e fix: use TryGet.optional 2021-07-26 15:33:36 +02:00
Christian Pauly e86fd7cc07 refactor: Restructure states box and use dedicated members hive box
More description about this here: https://gitlab.com/famedly/company/frontend/frontend-issue-inbox/-/issues/40
2021-07-10 08:38:38 +00:00
Christian Pauly aeb808b5dc fix: Dont migrate from version null
If the currentVersion of the database is null then the database has never been used yet.
Therefore we store the current version and do not call the migrate method.
2021-07-10 08:38:38 +00:00
Sorunome 4735d2d0a8
fix: Sending of the to_device key
With the switch to hive a regression of sending the to_device key was
introduced: When popping elements .deleteAt(), so deleting at the index,
was used, instead of .delete(), so deleting of the key. As the new events
pushed onto the queue used hives auto increment key, a .delete() is
appropriate here.
2021-07-09 19:43:48 +02:00
Christian Pauly 7c0dd5d8fd fix: Broken nested accountData content maps
If you call BasicEvent.fromJson the given content is copied first
which recursively makes sure
that the Map is from type
Map<String,dynamic>.
Using just the constructor doesnt have this which can lead that nested Maps in
the content is InternallinkedHashMap and
therefore lead to type errors.
2021-07-06 08:11:30 +02:00
Christian Pauly 96d28a1b76 feat: Implement migration for hive schema versions 2021-06-20 10:31:17 +02:00
Christian Pauly a7ffe4835d feat: Also migrate inbound group sessions
For users who do not use the chat backup this could be useful.
2021-06-19 11:03:52 +02:00
Christian Pauly 4dc62f6150 fix: Make sure hive keys only contain valid characters
There are some matrix IDs like room IDs which contain for example emojis.
This is valid from the spec and some people hack their Synapse to have this.
2021-06-19 10:51:17 +02:00
Christian Pauly c0c7f68bf3 chore: Rename to matrix 2021-06-18 09:58:25 +02:00
Christian Pauly 38d81d6093 fix: Migrate missing device keys
Currently we only migrate the client and SSSSCache but this leads to the
problem that we are no longer self signed after the migration.
We need to migrate all device keys too.
This also abstracts the migration code in a method. init() is too large already...
2021-06-18 09:16:12 +02:00
Christian Pauly 14ee16fe16 feat: Implement new Hive Database
The hive database now implements the whole API except for storing files which
should be better done by the flutter_cache_manager package inside of the
flutter app. All tests already run with Hive now but the Moor database is still
tested too. We needed to change some wait jobs in the tests because the Hive
database is not 100% in memory for the tests like Moor.
For now both database implementations are equal and the developer can pick
which one to use but we plan to get rid of Moor in the future.
2021-06-16 08:43:33 +02:00
Christian Pauly c387b33e9f refactor: Make support for file storing optional in database API 2021-06-01 15:27:18 +02:00
Christian Pauly 26261ecd5e feat: Add fromJson method for Room
This also included to move the room summary to a dedicated object inside
of the Room object because so it is easier to convert this to and from json.
2021-06-01 12:54:16 +02:00
Christian Pauly c3d47b16c6 CI: Use correct image 2021-05-25 10:46:34 +02:00
Christian Pauly c66e35c16c refactor: Create a clean database API 2021-05-20 07:36:03 +00:00
Lukas Lihotzki ac9df588d1 chore: recreate database.g.dart 2021-04-12 14:49:03 +02:00
Lukas Lihotzki 08043c06f4 refactor: high-level loops 2021-04-01 12:51:45 +02:00
Christian Pauly 0ceb2b26df refactor: Constants names 2021-03-30 12:43:50 +02:00
Sorunome c76dc0daa2
chore: Reduce needed database accessess 2021-03-09 18:28:58 +01:00
Krille Fear 44ca96e544 refactor: Update pedantic
This enforces to NOT use curly braces in Strings where not needed
and not await non-future variables.
2021-03-09 17:28:04 +00:00
Christian Pauly c9d3c327f6 refactor: Deprecate eventType in EventUpdate 2021-02-26 13:06:02 +01:00
Christian Pauly 72a7bc1637 feat: Implement upload sync filters 2021-02-18 14:03:58 +01:00
Sorunome d373a06aa2
fix: Add to_device queue to prevent olm session corruptions 2021-02-08 14:57:19 +01:00
Sorunome a35266f1e4
feat: Replay last sent olm message on olm session recovery from other device 2021-01-20 12:50:52 +01:00
Christian Pauly 9ffecd845a chore: Update license header 2021-01-04 10:34:50 +01:00
Christian Pauly e450a09fb5 refactor: External matrix api 2021-01-04 10:26:29 +01:00
Sorunome add19c73a0
fix: Load multiple olm sessions from the database at once for increased performance 2020-12-30 12:52:54 +01:00
Sorunome 6723c01a31
feat: Lazy-send room keys, chunked and sorted by importance 2020-12-29 14:46:36 +01:00
Marcus Hoffmann 54fc29f203 clear cache fix 2020-12-22 17:34:50 +01:00
Sorunome df1c249011
feat: Auto-Share megolm sessions with other users we know for a fact are allowed to see said message 2020-12-21 14:41:44 +01:00
Christian Pauly 3d6dbc9dc3 chore: Implement logoutput 2020-12-21 09:35:36 +01:00
Christian Pauly 57286d4c28 feat: Use logger package 2020-12-19 12:04:25 +00:00
Sorunome 49f0679fbf feat: Add bootstrapping 2020-12-10 08:13:24 +00:00
Christian Pauly bec1761172 refactor: content parsing 2020-11-30 15:34:46 +01:00