Why the baseline is the whole argument
Every performance complaint about an app is a comparison, and most of the time nobody has the before half of it. Without a baseline you cannot separate an app that added 300 milliseconds from a theme change made the same week, from a platform update, or from the fact that the first test ran on hotel wifi.
Capturing one takes about ten minutes. Do it before you install anything, repeat it after, and the conversation stops being about impressions.
The four pages
Your best selling product page, because that is where an app block usually goes and where the money is. The home page, because it is the page people will quote at you. The collection page most of your ads point at. And the cart, because cart timers, bundle and upsell apps do their work there and it is the page merchants forget to test.
Save the exact URLs somewhere you will find them again. A baseline measured against a URL that no longer exists is not a baseline.
What to record for each one
A mobile Lighthouse run in an incognito window with extensions off, three times, keeping the middle result. Write down Largest Contentful Paint, Total Blocking Time, Cumulative Layout Shift, and the transferred bytes. Export the JSON as well, it is one click and it settles detail arguments later.
A full page screenshot at phone width, which catches layout that numbers never will. A month from now you will want to know whether the buy button used to sit that far down the page.
A list of what is installed, taken from the apps page in your admin, plus the theme name and version. Both of those change, and both explain results that otherwise look like magic.
Repeat it properly
After the install, run the same four pages the same way, on the same connection, ideally at the same time of day. Then change one thing at a time. If you install three apps in an afternoon, you have measured an afternoon.
Keep everything in one folder per date. Two dated folders beat any amount of recollection, and they are what makes a support conversation short and unemotional.
Re-run the baseline on a schedule. Once a quarter is plenty, plus once before any theme change. Stores get slower gradually, one small addition at a time, and a quarterly file is the only thing that makes gradual visible.
Sharing it with an app developer
A useful report is short and comes in pairs: the URL, the two Lighthouse JSON exports, the date and time of each run, the app version, and one sentence saying which number moved and by how much. That is enough for whoever wrote the code to reproduce it, and it usually earns an answer the same day rather than a request for more information.
Leave the conclusions out. Saying that a block adds 400 milliseconds to Largest Contentful Paint is a measurement, and it is actionable. Saying the app is slow is an opinion, and the reply will be an opinion as well. The pair of files does the arguing for you and keeps the exchange short.
On the same subject
Reading a Lighthouse report without a developer. Which four numbers in a Lighthouse report matter to a merchant, which ones are noise, and how to run the test so that the second result means something.