In 2016, I launched the primary model of the device, then referred to as not Statoscope, however Webpack Runtime Analyzer. That was a technical demo of one other rempl device made by Roman Dvornov.
We needed to offer a visible interface to research bundles in real-time. That’s, you run the particular device in your browser, and it reveals you what is going on inside your bundle: what it consists of, is it working now, which errors have occurred. At first, utilities for these duties have been obtainable within the console, however not within the browser.
After a 12 months of labor on the mission I ran out of concepts, so I put it on pause. However at about the identical time, I began working extra carefully with webpack, contributing to its core. So, in 2018, I needed to revive Webpack Runtime Analyzer, utilizing my newly gathered expertise.
In October 2020, I launched the primary model of Statoscope. Mainly, it’s the identical as Webpack Runtime Analyzer however in a distinct packaging and with extra refined options. It’s merely a extra elaborate, in-depth device for analyzing your bundle.
Content material Evaluation
This is likely one of the fundamental options of Statoscope. However first, let’s see how bundling works and what it encompass.
Here’s a fairly typical configuration with two entry factors:
module.exports = {
entry: {
fundamental: ‘./fundamental.js’,
admin: ‘./admin.js’,
},
}
That is the principle web page and admin space. Nothing particular, apparently.
Once we run the bundling, webpack defines every JS- or TS-file as a module. The bundle is made up of many JS modules (the yellow squares within the illustration above).
The modules are fashioned into teams — so-called chunks (the blue squares). There’s a gaggle of modules for the principle web page, one other for the admin space, and so forth (the blue squares).
All that’s wrapped up within the belongings (the crimson squares). These are the output recordsdata, which you see within the dist listing (or construct, relying on how yours is ready up). That is what webpack operates with.
Let’s try the phases of the bundling, so we will higher perceive what’s taking place.
Wepback takes JS-, TS- or CSS- modules as enter and parses them into an summary ‘syntax tree’.
Subsequent, Webpack defines hyperlinks between the modules. For instance, Module A imports Module B. And Module B exports one thing. Mainly, a graph of dependencies between the modules is created.
Then, Webpack optimizes this graph. It may not be fully environment friendly if we use modules in uncooked kind, as they’re within the file system: there is likely to be some doubles, some modules may very well be mixed, and others are solely partially used.
One other instance: a module has two exports, however we solely use one. We don’t want the second export, so we will painlessly take away it with out harming the bundled code. That is all a part of Webpack’s optimization course of, with a view to cut back the dimensions of output belongings.
Webpack then generates the content material of the recordsdata that may find yourself within the dist listing. That is the rendering of the belongings.
Lastly, these recordsdata are merged into the listing.
These are the fundamental phases of how Webpack operates, and what it does with these modules.
What Does Statoscope Work With?
Webpack offers Statoscope with details about all of the modules, chunks and belongings that it encounters through the construct course of. They’re supplied within the type of so-called stats — a giant stats.json file, which Webpack generates along with the bundle. This file is transferred to Statoscope (truly, more often than not Statoscope takes it itself). Statoscope reads the file and generates a report within the browser which we will have a look at, analyze, and so forth.
Briefly, right here’s the method: Webpack generates stats and sends them to Statoscope, which then generates an HTML report primarily based on them.
How To Obtain stats?
There are two methods. First, you possibly can run webpack with –json argument and specify during which file to avoid wasting the stats. Then you possibly can add the stats to the sandbox website, which I arrange particularly so that you can simply drop your stats there and analyze them. That is the best approach to get a Statoscope report with out integrating Statoscope wherever:
$ webpack –json stats.json
Nevertheless, I like to recommend the second manner, as a result of it collects far more details about your bundle. You simply have so as to add the Statoscope plugin to your webpack configuration.
config.plugins.push(new StatoscopeWebpackPlugin())
That’s it! Simply add the Statoscope plugin to your webpack configuration.
What Are stats?
{
“modules”: [/*…*/],
“chunks”: [/*…*/],
“belongings”: [/*…*/],
“entrypoints”: {/*…*/},
/*…*/
}
Stats are the inner info from webpack: which modules, JS-, TS-, and CSS-files it got here throughout, into which chunks these recordsdata have been mixed, and the ensuing belongings. All that is merged into one massive .json file, which Statoscope then analyses.
Particulars
Let’s have a look at some module info in additional element:
{
“identifier”: “babel-loader!./src/index.js”,
“title”: “./src/index.js”,
“causes”: [/*…*/],
/*…*/
}
For instance: Our fundamental utility file, index.js, has an identifier and file title as such; it’s positioned on this path. Within the causes property, we will see the place this file is imported to, and the place it’s used. That is the form of info that Webpack offers within the stats.
Statoscope processes the stats, performs evaluation and offers a visible illustration to interpret the leads to a handy kind:
All of the issues I’ve talked about above are right here: entrypoints, modules, chunks and belongings.
Within the center picture above, we will see the chunk fundamental that’s async.
There are typically two varieties of chunk:
preliminary
This one should load when the web page is loaded.
async
This one is the results of a dynamic import, and may not load instantly on the web page; therefore the title, asynchronous. In case you break up your utility into dynamic elements — for instance, if in case you have a big library for animations that solely must load when the animation is required — that is an asynchronous chunk.
Copies Of Packages And Their Variations
Based mostly on this similar info, Statoscope additionally calculates the bundle tree — principally the npm-package which is utilized in your bundle: which packages and what number of copies of every.
You might need a scenario if you arrange, for instance, Package deal A and Package deal B. These two packages each use the identical Package deal C — however completely different variations. It seems that abruptly a double of Package deal C seems in your bundle. Statoscope tells you: “You could have two variations of the bundle fbjs. On the root, model 0.8.17 is used, whereas model 2.0.0 is used someplace in draft-js. Please resolve this. Replace your dependencies, and the bundle duplicate is gone.”
As you possibly can see, Statoscope enriches the stats with some extra info: for instance, about bundle variations. Initially, Webpack stats don’t embrace details about bundle variations. Later I’ll clarify how this occurs.
Module Map
In case you’ve already used different instruments for analyzing stats, eg. Webpack Bundle Analyzer, then you ought to be conversant in the module map.
The module map is one a part of Statoscope. The distinction is that Webpack Bundle Analyzer analyses the internals of Webpack relatively than stats. The concept is principally the identical, however the approaches are completely different. Statoscope means that you can analyze modules, module maps, chunks and belongings — multi function place.
We will open a bundle web page — for instance, asn1 — and see what it consists of. Wish to know why the bundle weighs a lot? Merely open a selected bundle web page and have a look.
Evaluating Bundles
We will choose earlier and present stats, click on the button and Statoscope will present what has modified, typically: which modules have been there, which have been created, eliminated, added or just modified (for instance, dimension change); which chunks have been added, modified, eliminated and so forth.
For instance, let’s say that the present stats are out of your present characteristic department. You’re making an superior characteristic, and the ‘earlier than’ stats may very well be stats from the grasp department. You wish to know: will my new characteristic crush the bundle? Have I made worse what’s presently within the grasp? You may ask these items to Statoscope, and it’ll let you know if it bought worse (or higher), and the way.
Customized Stories
If Statoscope is just not offering you with sufficient studies, then you possibly can generate your personal studies.
However there are a couple of issues that Statoscope may resolve.
Stats are large .json recordsdata, which might develop into a number of gigabytes massive. This file is its personal format, and to extract information it’s typically vital to write down a number of code. In all probability, no person desires to write down that code, particularly if there’s so much. If solely there have been an easier manner…
Jora
One resolution to this downside is a question language referred to as jora.
Right here’s some JS code that we will use to extract from the stat a listing of modules and kind them out by title:
const modules = [];
for (const compilation of compilations) {
for (const module of compilation.modules) {
modules.push(module);
}
}
modules.kind((a, b) => a.title.localeCompare(b.title))
And right here’s a bit of piece of code in Jora, which does the very same factor.
compilations.modules.kind(=>title)
We are saying: take all of the compilations, and take all of the modules from them, and kind them by title. That’s it!
As you possibly can see, Jora permits us to scale back the quantity of JS-code we write. It’s a lightweight, semantic, comprehensible language for queries in JSON.
In case you have already labored with libraries like jq, then you definitely’ll be conversant in it.
Listed here are three examples from Jora:
Filter
modules.[size > 1000]
We will say: give me all of the modules that are greater than a thousand bytes.
Map
modules.({module: $, dimension})
We wish to convert modules into one other object. We are saying: give me modules and add sizes to them. We’ll now have an object with two fields: module and dimension. Simple and concise.
Map Name
modules.(getModuleSize(hash)).dimension
We make a map, and to get the ensuing object we additionally generate a operate that returns the thing to us.
UI
So, we’ve labored out how Statoscope lets us make stats queries and create studies utilizing the question language Jora. However how you can show it? We’d like some UI — we will’t simply show the construction of the JS object. We wish to add the listing, title, buttons, and badges.
Looks like we will make the UI in React. However then we run into extra issues: the interface needs to be bundled and hosted someplace. Statoscope offers a approach to resolve this downside.
Discovery.js is a platform for declarative UI.
Examine this instance. On the left, you possibly can see the premade UI parts. Discovery.js accommodates a ready-made UI package: buttons, titles, badges, indicators and so forth. That’s all prepared so that you can use.
However the principle characteristic is that you would be able to describe ‘structure’ (views composition) with JSON.
That’s, we are saying: I would like an indicator, the label and the worth might be as such.
Discovery.js will create an HTML view as an alternative of the JSON. Making a composition from such JSON objects, you possibly can create your personal studies.
With the assistance of jora, you obtain information; with ‘structure’ in Uncover.js you get a view. Put them collectively, and also you get a report.
Placing It All Collectively
Right here’s the way it appears to be like:
In case you click on the Make report button in Statoscope, then on the prime you possibly can enter any question in Jora, within the subject beneath you possibly can enter the UI — that’s, describe the structure within the JSON kind, and beneath that you just get your report.
On this instance, we take the modules from all of the compilations and kind them by dimension. We get a brilliant easy report. The heaviest modules might be on the prime.
An additional perk is that we will share a hyperlink to the report with colleagues. For instance, you ran a report in CI, analyzed it, and understood that one thing wasn’t working proper. Merely create the report, copy the hyperlink instantly in CI, and ship it to your colleague. Your colleague will open it and see precisely the identical report as you. So easy.
That is one approach to generate customized studies on the go. We clicked ‘Make report’, wrote a jora question, used Discovery.js structure, and voila: bought a report, good to go.
Integrating Into UI
Customized studies like these may also be embedded into your studies in CI, utilizing the Statoscope plugin for Webpack. This plugin has the property studies, which you inform: embed my customized report into my HTML report. That is virtually the identical because the earlier variant, however right here we now have solely the highest 20 heaviest modules; we lower off the decrease border.
new StatoscopeWebpackPlugin({
studies: [
{
id: ‘top-20-biggest-modules’,
name: ‘Top 20 biggest modules’,
data: { some: { custom: ‘data’ } }, // or () => fetchAsyncData()
view: {
{
data: `#.stats.compilations.(
$compilation: $;
modules.({
modules: $,
hash: $compilation.hash,
size: getModuleSize($compilation.hash)
})
).sort(size.size desc)[:20]`,
view: ‘listing’,
merchandise: ‘module-item’,
},
},
},
],
})
The title of the report is on the fifth line, the info on the sixth — if we’d like it. We’d want it, for instance, if we wish to create a customized report with metrics about how our bundle has modified. We will import info from our metric storage into information. For instance, we will document the each day common bundle construct time, ship it to storage with the metrics after which embed it into the customized report. We are saying: take the metrics from there and create a UI report with a line graph displaying how our bundle has modified over time. Discovery.js may create charts like this. So handy.
The studies you embed into the UI report seem in a dropdown menu like this:
You may share this report with colleagues. Keep in mind the distinction: Within the first case, you create your report on the go and share it with colleagues; on this case, you embed your report within the HTML, which you get in CI. This gained’t be a hyperlink with the request, view and so forth coded into the GET parameters; this report might be genuinely embedded into the HTML.
Validation
This can be a very cool, crucial factor. Why is it so essential to me? I’ve been attending to grips with it for weeks, making an attempt to work out how you can validate stats. I wish to set it up in order that our pull requests don’t find yourself within the grasp in case they worsen the standard of our bundle (e.g. by growing the bundle dimension or construct time). The grasp doesn’t want that.
Learn how to make the validation course of straightforward? I had a go searching, however I couldn’t discover any easy options.
I attempted other ways to give you one thing myself, making an attempt and failing, making an attempt and failing … however lastly I labored it out. That’s how Statoscope CLI was born. It’s a console utility that means that you can validate stats.
Set up CLI
npm set up -D @statoscope/stats-validator-plugin-webpack
Set up the plugin for the validation of webpack bundles
npm set up -g @statoscope/cli
How does it work? I didn’t wish to invent one thing fully new, so right here’s an ESLint which might be acquainted to you. It permits you to discover and repair issues in your JS/TS recordsdata. If one thing goes flawed, like our code doesn’t adjust to the foundations within the ESLint config, we’re notified of the error within the console. If the checks fail in CI, it’s important to go and repair them.
I needed to do one thing comparable in order that I wouldn’t need to reinvent the wheel. There are already loads of cool issues like Jora and Discovery.js. So, we determined to make use of the console utility Statoscope CLI in addition to the webpack-stats validation plugin (for webpack stats to be validated).
For the second, Statoscope solely works with webpack stats, however ideally, it is going to be a platform for any bundler. That’s why I made effort to architecturally sculpt each bit of the toolkit (it wasn’t for nothing that mentioned it was a toolkit) in order that it capabilities as a plugin, not as one thing hardcoded. And I’m getting there, step by step.
So, we set up Statoscope CLI and the webpack plugin for validating exactly webpack stats. In case you have Rollup, then sooner or later you’ll have a plugin exactly for its stats, or and likewise for esbuild stats. It’s like in ESLint: if you wish to verify TypeScript recordsdata, you put in the corresponding plugin; if you wish to verify one thing else, you put in a separate plugin and configure the foundations.
That is how the Statoscope config for validation appears to be like (truly, not just for validation):
validate: {
// use a plugin with webpack-specific guidelines
plugins: [‘@statoscope/webpack’},
reporters: [
// use console reporter
‘@statoscope/console’,
// use reporter that generated an HTML-report with validation results
[‘@statoscope/stats-report’, {open: true}],
].
guidelines: {
// a rule that fails validation if construct time is worsen no less than 10 seconds
‘@statoscope/webpack/build-time-limits’: [‘error’, {global: 10000}],
// you maight use every other guidelines
}
}
}
Right here, every thing appears to be like like ESLint — once more, no have to reinvent the wheel. We have now plugins for validating stats with.
We have now reporters to report the results of validation. For now, Statoscope has two built-in studies: a console one (which outputs every thing to the console) and stats-report (which generates studies). Producing studies is a killer characteristic of validation, as a result of looking for and analyzing error messages in massive numbers is tough.
Subsequent, I wish to join these error messages to particular modules. I wish to open a module and see which messages are associated to it. Doing this within the console is not possible: the textual content format is just not interactive, and I don’t wish to do any actually difficult UI stuff.
Within the guidelines property, we are saying which guidelines we wish to apply to the validation. There are about 12 guidelines now. For instance, there are guidelines for budgeting the bundle dimension, the preliminary bundle dimension, and the bundle loading dimension on the client-side.
For instance, you don’t need pull requests — which will increase the obtain time of your bundle on sluggish 3G — to finish up within the grasp. So, you set the rule, which Statoscope already has.
You may go to Statoscope’s repository and discover documentation on each rule. There are guidelines for a lot of numerous conditions, for instance “don’t allow utilizing bundle copies”. In case you occur to make a pull request that provides a double, this request is not going to be accepted, and you can be despatched again to do it once more (this type of factor occurs in CI).
Working Validation
It’s all fairly easy. We run this within the console:
$ statoscope validate –input ./stats.json
We enter the command statoscope validate, which lets us validate, and we point out on which stats-file we wish to run a check. We obtain a report (within the console and the browser).
Right here we now have validated one stats file. However along with a rule that analyzes a single stats file (like “are there doubles?”), can we make a rule that compares, for instance, how a lot you have got elevated the bundle dimension? For instance, you wish to improve not more than 2%. For that, it’s worthwhile to use stats from each the present department and the grasp.
So, a bit of terminology:
enter
A present stats file (out of your department).
reference
The earlier stats file with which the reference is made (from grasp).
Receiving A Report
Let’s have a look at what we noticed within the console, however in a digestible format:
This can be a tree the place every thing is highlighted, parts may be hidden or revealed, they are often hyperlinks (you possibly can verify the error message, or with which error it’s associated). Right here you possibly can even filter utilizing the sector above. That is far more handy than utilizing grep-like instruments and analyzing the console output.
Stats Queries
Transferring on to the following half relating to CLI. Why may we’d like stats queries?
Right here’s your stats.json: details about modules, chunks and belongings. We wish to get from the stats the dimensions or size of your bundle. You may want this, for instance, for creating customized feedback to your pull request or for sending metrics.
In case you have entry solely to the visible half, it turns into an issue: it’s not clear how you can extract the info from there. However you have got the console utility statoscope question, with which we will work all of it out. It lets us make a Jora question.
Below the inexperienced line: which question we wish to make; below the blue: which stats we wish to question. For instance, we are saying: give us the variety of modules. And the utility responds.
We will make any Jora question and put it aside in a separate file. So, we will say: give me the question from question.jora file, ship it to the statoscope question, apply it to those stats, after which save the consequence within the consequence.json file.
For instance, we wish to connect the variety of validation errors to the commentary of the pull request — please!
CI — Placing It All Collectively
Let’s see what occurs if we carry collectively validation and statoscope question. I’ll present this primarily based on Github Actions.
Commits In Grasp
As I’ve already mentioned, we now have input-stats (within the present department) and reference-stats (from the grasp). How can we acquire and retailer them? For each commit within the grasp, we make a bundle (if we dedicated one thing to the grasp, it signifies that every thing is okay). We get the stats from it, name the file reference.json and ship it to our storage of artifacts — which may very well be something.
If we’re speaking about GitHub, there’s already built-in artifact storage. We don’t have to do something particular, simply say add artifact. From then on, you have got stats from the grasp and they’re saved someplace.
Commits In Pull Requests
However most curiously: we even have feedback in pull requests. For every commit in a pull request, we construct the bundle, get the webpack stats from it, name that file enter.json, obtain the artifact reference.json from the earlier step, and also you now have two recordsdata. We give these two stats to statoscope validate (we get a report.html) and statoscope question (we get a consequence.json).
That’s it! We have now a UI report and customized information from the question, and we will now create (for instance) some customized commentary on behalf of our bot.
How can we do that? Let’s see. The commentary textual content might be some information from the question, plus some form of template. I’ll give a Jora question for instance — it’s not so long as it might sound:
It will get the construct time from two stats (reference and enter), subtracts one from the opposite and will get the distinction — for higher or worse. The identical factor occurs with preliminary dimension: how a lot our bundle has gotten greater or smaller. It additionally appears to be like on the variety of validation errors and dumps it in JSON.
Right here’s our commentary template:
Observe: I selected the template engine moustache, nevertheless it may very well be every other.
We are saying: the construct time modified like this, the preliminary dimension like this, the variety of validation errors are like this. And a hyperlink to the complete report.
So, right here’s what we find yourself with: we connect a commentary to drag requests on behalf of our bot.
We do all this as a part of our course of, our CI. We have now a variety of checks like checking our code, and in a separate step we now have the Statoscope verify: validations, downloading artifacts, our personal queries and creating commentaries. These are separate steps. On this case, we now have a crimson cross subsequent to the Statoscope verify which suggests one thing went flawed in validation.
You wish to know what? Merely go to the UI report from the commentary and luxuriate in.
The complete supply code Github Motion, which does every thing I’ve proven (with templates and Jora queries) you possibly can see within the statoscope.tech repository. This isn’t some ephemeral factor that I simply confirmed; every thing already works, for each pull request in that repository a report might be generated. You may enter a check pull request and see what occurs. (The supply code of each Github Actions may be discovered throughout the workflows file, so you possibly can see the way it all works.)
Plans
Coming to the tip of this text, I’ll let you know about our upcoming plans as a result of they’re large.
Customized stats format.
Do not forget that I’m taking Statoscope in direction of independence from any particular bundler. However so as to not block it off from completely different stats codecs, I wish to remodel all of them into one common format.
I actually wish to allow you to lengthen the UI.
Typically, extensibility is a key particularity of the mission, which I’m working in direction of. I would like Statoscope to be prolonged with plugins.
The above-mentioned help from different bundlers.
I wish to decouple Statoscope from webpack, in order that no point out of webpack is made in Statoscope — solely within the plugins.
Less complicated course of.
I wish to mix the mixing course of in CI right into a single Github Motion and publish it as a bundle, so that you just don’t copy and paste the supply code however relatively set up Github Motion and use it.
A single portal for documentation.
In the intervening time all documentation is shoved into readme recordsdata within the repository. In case you go to the repository, you’ll see the packages folder, and there’s principally some documentation for some packages. I would like a single portal for documentation — one thing like how Jest have achieved it (I really like the best way they’ve achieved it).
I wish to help rempl (a expertise which appeared in demo kind as Webpack Bundle Analyzer in 2016).
The concept is that for those who use a bundle in a watch-mode like HMR (Sizzling Module Substitute), you open your browser and also you’ll have entry to Statoscope instantly within the developer instruments within the browser. You alter one thing within the supply code (like properties of some React element), you put it aside, HMR works it out for you, Statoscope is within the develop instruments of your browser as properly, and also you don’t have to generate studies continually.
Evaluation of the bundler config.
Configuration, particularly webpack, is after all a separate story and a separate talent. And I actually wish to present colleagues with some recommendation and modify the config in actual time, in order that it’s efficient. When Statoscope has performance like this, it is going to be simpler to make efficient bundles.
Suggestion on the optimization of bundles.
I would love Statoscope to have the ability to say: “Hear colleague, you’ve bought this factor flawed, you possibly can optimise one thing right here — make these modifications, and also you’ll cut back the dimensions of your bundle by this many megabytes.”
Redesign.
I wish to give particular because of my colleague Danil Avdoshin, who discovered the free time to work on the redesign. For now it’s solely saved in Figma prototypes, however I’m positive that we’ll get spherical to ending it quickly.
And my private dream: bundle score.
I would like to have the ability to examine the standard of bundles. For now, I’m simply fascinated about how to do that, however I wish to see the highest bundles by way of configuration effectivity, the effectivity of utilizing webpack potentialities, and so forth.
Why am I telling you about my plans? Statoscope is a totally open-source mission, so I welcome points and pull requests. I shortly reply to points. If you wish to take part in a critical mission, you possibly can assist with any of the factors above.
Right here’s how Danila designed the way it might look:
The purpose right here is the versatile UI: you possibly can customise which blocks you want on the web page, and determine typically what you wish to present. For instance, I don’t want a listing of modules, however as an alternative I can present a chart in that area.
Danila designed this prototype, it’s in Figma, you can too contribute right here. I’ll get there finally, however in the meantime, you possibly can assist.
Name To Motion
Lastly, what I’m asking for on this article:
Strive statoscope.tech. It’s a sandbox, there’s demo information, and you’ll load your personal.
Strive @statoscope/webpack-plugin. Keep in mind, this plugin collects far more info than webpack itself offers.
Strive @statoscope/cli for validation, queries and creating commentaries. By the best way, Statoscope is presently utilized in Andrei Sitnik’s size-limit bundle. Not way back we built-in it there collectively, changing Webpack Bundle Analyzer. Now if you run size-limit with the –why (“Why did the dimensions improve”), Statoscope will open.
Share suggestions, ask questions.
Get conversant in Jora. It’s a easy, concise language with out too many constructions. And also you’ll have the ability to write studies no less than, after which guidelines.
Check out Discovery.js.
Write your report.
Write your validation rule.
Write your permission.
Deliver your PR/Situation.
Put a star on GitHub. I’d like that. 🙂
Subscribe to MarketingSolution.
Receive web development discounts & web design tutorials.
Now! Lets GROW Together!