Offline support for Network page #8332

hrajwade96 · 2024-09-21T09:19:46Z

Adding offline support to Network page.

issue link - #4470

List which issues are fixed by this PR.

Please add a note to packages/devtools_app/release_notes/NEXT_RELEASE_NOTES.md if your change requires release notes. Otherwise, add the 'release-notes-not-required' label to the PR.

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read the Flutter Style Guide recently, and have followed its advice.
I signed the CLA.
I listed at least one issue that this PR fixes in the description above.
I updated/added relevant documentation (doc comments with ///).
I added new tests to check the change I am making, or there is a reason for not adding tests.

If you need help, consider asking for help on Discord.

This reverts commit 0c1f414.

kenzieschmoll · 2024-09-30T21:30:15Z

@hrajwade96 is this PR ready for review or is this still a work in progress?

hrajwade96 · 2024-10-01T17:13:00Z

@hrajwade96 is this PR ready for review or is this still a work in progress?

@kenzieschmoll just finishing up few things, I will mark it ready soon

…into network_screen_offline_support

kenzieschmoll · 2024-10-02T23:23:08Z

packages/devtools_app/lib/src/screens/network/network_screen.dart

-    unawaited(controller.startRecording());
-
-    cancelListeners();
+    try {


instead of doing this logic in the screen, we want to do this in the screen's controller NetworkController. Checkout the guide in the dart doc for OfflineScreenControllerMixin. This provides pretty thorough documentation on how offline support for a screen should be set up: https://github.com/flutter/devtools/blob/master/packages/devtools_app/lib/src/shared/offline_data.dart/#L66-L124.

You can also use the ProfilerScreenController as an example: https://github.com/flutter/devtools/blob/master/packages/devtools_app/lib/src/screens/profiler/profiler_screen_controller.dart/#L81

not sure how I missed it, apologies. I have fixed it now and scanning through if I have missed anything.

Can you confirm if the changes are correct?

kenzieschmoll · 2024-10-02T23:33:02Z

packages/devtools_app/lib/src/screens/network/network_controller.dart

+        filteredData.value.whereType<DartIOHttpRequestData>().toList();
+    return OfflineScreenData(
+      screenId: NetworkScreen.id,
+      data: serializeRequestDataForOfflineMode(requests),


I recommend creating a class OfflineNetworkData with Serializable that packages up all of the data stored on the Network controller for serialization. This includes the list of requests as well as other information like the currently selected request. In the future we may want to include the current filter, but let's leave that out for now.

Creating a new class OfflineNetworkData will make it easier to test the toJson and fromJson methods that should be implemented on the class. See other examples of classes that mixin Serializable in DevTools for an example.

Then here, you'd create an instance of OfflineNetworkData from the current data on the network controller, and call toJson() to get the value for the data parameter of OfflineScreenData in this return statement.

kenzieschmoll · 2024-10-07T17:12:17Z

packages/devtools_app/lib/src/screens/network/network_controller.dart

@@ -160,6 +167,58 @@ class NetworkController extends DisposableController
  @visibleForTesting
  bool get isPolling => _pollingTimer != null;

+  void _initHelper() async {
+    try {


why do we need the try / catch here?

done, removed it.

packages/devtools_app/lib/src/screens/network/network_controller.dart

kenzieschmoll · 2024-10-07T18:05:16Z

packages/devtools_app/lib/src/screens/network/network_controller.dart

+          shouldLoad: (data) => !data.isEmpty,
+          loadData: (data) => loadOfflineData(data),
+        );
+      }


the code after this if statement should be put in an else block

not using else was an oversight, added else block now, and also now there is no code outside.

kenzieschmoll · 2024-10-07T18:05:49Z

packages/devtools_app/lib/src/screens/network/network_controller.dart

+          loadData: (data) => loadOfflineData(data),
+        );
+      }
+      cancelListeners();


you can remove this line since this initialization code should only be run once. When this was in the widget code, we needed to call cancelListeners because didChangeDependencies may be called more than once.

Ok got it, removed

packages/devtools_app/lib/src/screens/network/offline_network_data.dart

kenzieschmoll · 2024-10-07T19:52:40Z

packages/devtools_app/lib/src/screens/network/network_screen.dart

@@ -187,7 +180,7 @@ class _NetworkProfilerControlsState extends State<_NetworkProfilerControls>
  @override
  void initState() {
    super.initState();
-
+    addAutoDisposeListener(offlineDataController.showingOfflineData);


you can remove this if you use OfflineAwareControls instead

kenzieschmoll · 2024-10-07T19:55:33Z

packages/devtools_app/lib/src/screens/network/offline_network_data.dart

+    final List<dynamic> requestsJson = json['requests'] ?? [];
+    final requests = requestsJson


we need to track the socket statistics as well.

Yes sure, will add it, I knew it is required but thought of first doing it for http requests to start with, and later add the implementation for socket reqs.

kenzieschmoll · 2024-10-07T19:56:10Z

packages/devtools_app/lib/src/screens/network/offline_network_data.dart

+      'requests': requests.map((request) => request.toJson()).toList(),
+      'selectedRequestId': selectedRequestId,


store these strings as static consts on this class or as an enum in this file

kenzieschmoll · 2024-10-08T16:43:55Z

packages/devtools_app/lib/src/screens/network/network_controller.dart

+        timelineMicrosOffset: DateTime.now().microsecondsSinceEpoch -
+            (networkService.timeStamp ?? 0),


You can just use DateTime.now().microsecondsSinceEpoch for this. This is arbitrary for offline data since we won't be updating the sockets like we do when we have a live connection.

kenzieschmoll · 2024-10-08T16:45:11Z

packages/devtools_app/lib/src/screens/network/network_screen.dart

+        OfflineAwareControls(
+          controlsBuilder: (_) => _NetworkProfilerControls(
+            controller: controller,
+            offline: offlineDataController.showingOfflineData.value,


use the value from the controlsBuilder instead of looking this up from the notifier again:

controlsBuilder: (offline) => _NetworkProfilerControls( controller: controller, offline: offline, ),

kenzieschmoll · 2024-10-08T16:50:35Z

packages/devtools_app/lib/src/screens/network/network_screen.dart

+        SearchField<NetworkController>(
+          searchController: widget.controller,
+          searchFieldEnabled: hasRequests,
+          searchFieldWidth: screenWidth <= MediaSize.xs
+              ? defaultSearchFieldWidth
+              : wideSearchFieldWidth,
+        ),
+        const SizedBox(width: denseSpacing),
+        DevToolsFilterButton(
+          onPressed: _showFilterDialog,
+          isFilterActive: _filteredRequests.length != _requests.length,
+        ),


only include these two controls in offline mode.

children: [ if(!widget.offline) ...[ StartStopRecordingButton(...) const SizedBox(width: denseSpacing), ClearButton(...) const SizedBox(width: denseSpacing), DownloadButton(...) const SizedBox(width: denseSpacing), ] Spacer(), // this is a helper widget that replaces Expanded(child: SizedBox()), const SizedBox(width: denseSpacing), SearchField<NetworkController>(...) const SizedBox(width: denseSpacing), DevToolsFilterButton(...), ],

Ok reverting this and adding Spacer.
Actually I had changed it again thinking you meant keeping all the controls, which I got clarified later.

kenzieschmoll · 2024-10-08T17:08:56Z

packages/devtools_app/lib/src/screens/network/network_controller.dart

+      currentRequests: _networkService.currentHttpRequests,
+      socketStats: _networkService.sockets ?? [],


We shouldn't have to make any changes to the _networkService to do this. I think what you want is this:

mixin Serializable to the NetworkRequest class. This will force subclasses of NetworkRequest to implement a toJson method.

Then implement toJson for the NetworkRequest subclasses DartIOHttpRequestData and Socket. You will also need to implement the deserialization logic fromJson on these classes as well if they are not already implemented.

Then here you can create offline data as follows:

final httpRequestData = <DartIOHttpRequestData>[]; final socketData = <Socket>[]; for (final request in _currentNetworkRequests.value) { if (request is DartIOHttpRequestData) { httpRequestData.add(request); } else if (request is Socket) { socketData.add(request); } } final offlineData = OfflineNetworkData( httpRequestData: httpRequestData, socketData: socketData, selectedRequestId: selectedRequest.value?.id, );

In OfflineNetworkData toJson and fromJson methods, you can use the serialization methods you implemented on the Socket and DartIORequestData classes.

Let me know if you have any questions on this.

This will give the reqs lists of - DartIOHttpRequestData and Socket classes.
I had implemented this for DartIOHttpRequestData, similarly I can do for socket, by using Serializable mixin on their parent abstract class.

However, you suggested to use the updateOrAddAll function

void updateOrAddAll({ required List<HttpProfileRequest> requests, required List<SocketStatistic> sockets, required int timelineMicrosOffset, })

Which requires below lists :
List<HttpProfileRequest> & List<SocketStatistic>.

And I think this data is only available in NetworkService, also there are no getters for this data in DartIOHttpRequestData and Socket classes.
Or I should create those getters and retrieve it from there, is that what you were suggesting?

The Socket class wraps a SocketStatistic object, so you can get the List<SocketStatistic> from the List<Socket>. I recommend adding this extension method to the bottom of the network_model.dart file so that you don't have to change the visibility of Socket._socket.

extension SocketExtension on List<Socket> { List<SocketStatistic> get mapToSocketStatistics { return map((socket) => socket._socket).toList(); } }

Also SocketStatistic already has a fromJson method you can leverage. I expect the serialization logic for Socket would be something simple like:

Map<String, Object?> toJson() { return { 'timelineMicrosBase': _timelineMicrosBase, 'socket': _socket.toJson() // you'll need to implement this, perhaps as an extension method on SocketStatistic }; } Socket fromJson(Map<String, Object?> json) { return Socket( SocketStatistic.fromJson((json['socket'] as Map).cast<String, Object?>()), json['timelineMicrosBase'] as int, ); }

Oh ok, I think visibility can remain intact even with :

SocketStatistic get socketStatistic => _socket;

But yes extension adds more utility here, will use it.

added initial implementation

fe11d5b

hrajwade96 requested review from bkonyi, kenzieschmoll and a team as code owners September 21, 2024 09:19

hrajwade96 marked this pull request as draft September 21, 2024 09:19

hrajwade96 changed the title ~~added initial implementation~~ Offline support for Network page Sep 21, 2024

hrajwade96 added 3 commits September 29, 2024 18:20

adding constants file

6c140b4

updated NEXT_RELEASE_NOTES.md

0c1f414

Revert "updated NEXT_RELEASE_NOTES.md"

e7eeecb

This reverts commit 0c1f414.

hrajwade96 and others added 2 commits October 2, 2024 22:46

used constants, added util for header size calc, added params

c5ec6f6

Merge branch 'master' into network_screen_offline_support

131cde6

hrajwade96 marked this pull request as ready for review October 2, 2024 17:19

hrajwade96 added 3 commits October 2, 2024 22:56

param renamed, used _log

2a9c4cb

Merge remote-tracking branch 'origin/network_screen_offline_support' …

4f473be

…into network_screen_offline_support

added copyright header

ae58738

kenzieschmoll reviewed Oct 2, 2024

View reviewed changes

hrajwade96 added 5 commits October 3, 2024 18:35

setup changes implemented

2bfbe1c

put class name in square brackets

857cfbe

added isNullOrEmpty check

7150436

added exit offline cta, hiding other controls, removed json string

da11a67

fix for request selection

941ec88

kenzieschmoll reviewed Oct 7, 2024

View reviewed changes

hrajwade96 added 5 commits October 8, 2024 09:09

comments resolved

852b144

removing getFullRequestData section

433af38

using updateOrAddAll for setting data

4644b00

Wrapped with OfflineAwareControls, removed if-else.

6fd98d0

removed addAutoDisposeListener as we are using OfflineAwareControls

665ea0d

removed try-catch

b6df572

kenzieschmoll reviewed Oct 8, 2024

View reviewed changes

hrajwade96 added 4 commits October 9, 2024 18:05

code refactoring

be63db4

added enum

91505b5

code refactoring, reverting changes on network service

0715379

using values from enum

ade8b5f

hrajwade96 requested a review from kenzieschmoll October 11, 2024 18:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Offline support for Network page #8332

Offline support for Network page #8332

hrajwade96 commented Sep 21, 2024 •

edited

Loading

kenzieschmoll commented Sep 30, 2024

hrajwade96 commented Oct 1, 2024

kenzieschmoll Oct 2, 2024

hrajwade96 Oct 3, 2024

hrajwade96 Oct 5, 2024

kenzieschmoll Oct 2, 2024

hrajwade96 Oct 3, 2024

hrajwade96 Oct 5, 2024

kenzieschmoll Oct 7, 2024

hrajwade96 Oct 8, 2024

kenzieschmoll Oct 7, 2024

hrajwade96 Oct 8, 2024 •

edited

Loading

kenzieschmoll Oct 7, 2024

hrajwade96 Oct 8, 2024

kenzieschmoll Oct 7, 2024

hrajwade96 Oct 8, 2024

kenzieschmoll Oct 7, 2024

hrajwade96 Oct 8, 2024

kenzieschmoll Oct 7, 2024

hrajwade96 Oct 8, 2024

kenzieschmoll Oct 8, 2024

hrajwade96 Oct 9, 2024

kenzieschmoll Oct 8, 2024

hrajwade96 Oct 9, 2024

kenzieschmoll Oct 8, 2024

hrajwade96 Oct 9, 2024 •

edited

Loading

hrajwade96 Oct 9, 2024

kenzieschmoll Oct 8, 2024 •

edited

Loading

hrajwade96 Oct 9, 2024 •

edited

Loading

kenzieschmoll Oct 9, 2024

kenzieschmoll Oct 9, 2024 •

edited

Loading

hrajwade96 Oct 9, 2024 •

edited

Loading

		final List<dynamic> requestsJson = json['requests'] ?? [];
		final requests = requestsJson

		'requests': requests.map((request) => request.toJson()).toList(),
		'selectedRequestId': selectedRequestId,

		timelineMicrosOffset: DateTime.now().microsecondsSinceEpoch -
		(networkService.timeStamp ?? 0),

		currentRequests: _networkService.currentHttpRequests,
		socketStats: _networkService.sockets ?? [],

Offline support for Network page #8332

Are you sure you want to change the base?

Offline support for Network page #8332

Conversation

hrajwade96 commented Sep 21, 2024 • edited Loading

Pre-launch Checklist

kenzieschmoll commented Sep 30, 2024

hrajwade96 commented Oct 1, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hrajwade96 Oct 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hrajwade96 Oct 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kenzieschmoll Oct 8, 2024 • edited Loading

Choose a reason for hiding this comment

hrajwade96 Oct 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kenzieschmoll Oct 9, 2024 • edited Loading

Choose a reason for hiding this comment

hrajwade96 Oct 9, 2024 • edited Loading

Choose a reason for hiding this comment

hrajwade96 commented Sep 21, 2024 •

edited

Loading

hrajwade96 Oct 8, 2024 •

edited

Loading

hrajwade96 Oct 9, 2024 •

edited

Loading

kenzieschmoll Oct 8, 2024 •

edited

Loading

hrajwade96 Oct 9, 2024 •

edited

Loading

kenzieschmoll Oct 9, 2024 •

edited

Loading

hrajwade96 Oct 9, 2024 •

edited

Loading