add _real_ stacked area charts [feature request]

[salim-b]

I'm filing this issue as a gathering point for the feature request of real stacked area charts.

The current solution to create stacked area charts is to plot cumulative variables which has multiple drawbacks:

• Correct (i. e. non-cumulative) hover values have to be set manually by using text: '...'.
• The resulting chart is incorrect if some traces are shown and others hidden (by clicking on legend items).

A real solution would be to have an argument like layout = {linemode: 'stack'}, same as there is for bar charts.

I'm aware that @etpinard stated some time ago:

We are planning on adding area charts to our list of trace types at some point this year. You are not the first person to ask for this feature.

Therefore I hope opening this issue won't be seen as an annoying harassment! Since it is an often requested feature, I think it important to have a place for users like me to gather all relevant information and potential progress on this.

Some more information:

• relevant bug reports:
  • plotly.js: Stacked Area Functionality #344
  • plotly for R: Example code of stacked fill area  plotly.R#686 and add _real_ stacked area charts [feature request] plotly.R#810
• Generating a stacked area chart in ggplot2 and then converting it to plotly using ggplotly doesn't work as supposed either (the plot doesn't get rescaled correctly if traces in between are unselected, there will just be space left).
• proposed workaround by @etpinard (in JS): https://codepen.io/etpinard/pen/yOgdOb

[treadmillian]

What are the timescales for this requested feature? This is one reason why I've turned to HightCharts which does this perfectly.

[Reubentrapdoor]

Been monitoring this one for a long time as well hoping to see some progress, all the workarounds I've seen are way too buggy to use in any production environment

[nicolaskruchten]

Here's a rundown of examples that show a reasonably well-specified way of dealing with misaligned x values, assuming we're stacking in the y dimension: https://docs.google.com/spreadsheets/d/1XZ2yEIN4S_Q-6c6AHr4F9LyKukQAcyg0gh-1VcumO9M/edit

[nicolaskruchten]

/cc @alexcjohnson

Excel:

Google Sheets:

Two different approaches to the drawing of lines where there is no data... i.e. in the second chart: Google Sheets draws a red line but Excel does not. Probably worth thinking carefully about the tradeoffs there, and ditto about what to show in hovers.

[alexcjohnson]

I don't think there's actually a difference between the two: Excel isn't drawing lines at all, only fills, so there's nothing to omit, but presumably you can turn lines on and then I bet they would be drawn the same way as Google's.

As @nicolaskruchten alludes to, the question of what to do with mismatched x values is the key sticking point here, the reason adding stacked area charts is not as easy for us as adding stacked bars.

Google sheets (and Excel, at least by default, I haven't looked in detail) has a simpler data model than we do: every series shares the same x data, so it's not possible to have mismatched x, the most you can do is have empty y at certain x values. They seem to treat those empties as zeros. That's certainly a plausible interpretation for certain data anyway, but not all, and it differs from how we handle scatter (line) in other contexts - where a missing y (or x for that matter) either leaves a gap or gets the line drawn straight from one valid point to the next, depending on the connectgaps setting.

Seems to me when we stack area charts we can internally fill in missing x values across all the stacked traces, and then there are perhaps three ways you might want to interpret gaps:

• Treat them as 0 (like google sheets does) - this would arise for example if your data came from doing an SQL aggregation (sum of sales by month, for example) and there were no events to aggregate in some of the periods.
• Interpolate (linearly?) across the gap. This is basically the equivalent of connectgaps: true, and would make sense in cases of incomplete data, for example you're summing populations across different countries by year, you don't have data for every country for every year but interpolating is a good assumption (certainly better than guessing zero population in those years!). This could get tricky if we want to take line.shape into account and try to make the (first) stacked trace look identical to its unstacked alternative. That's probably not necessary though, at least to begin with we can interpolate linearly (which is also probably about the only interpolation option that preserves the total, independent of stacking order). But I think it probably is important to not display markers at the interpolated points.
• Leave a gap. If you really want the plot to show what's truly known, with no extra interpretation, then gaps should be left empty. This is equivalent to connectgaps: false. We could do complicated things with stacking gap-less traces on top of gapped traces, but since the point here is to not make any assumptions that aren't explicit in the data, probably we'd want all gaps to propagate upward to the top of the stack.

So if the second and third cases are covered by connectgaps, what about the first (which, to fit with Google & Excel, should be the default)? I suppose it could be a new connectgaps: 'zero' or something? There would also be an argument for making this a separate setting, so that x values with an invalid y ('', null, non-numeric) would be treated differently from x values that get inserted just because they're present in other data sets. Perhaps you'd like newly-inserted x values to get y=0 but invalid y to be treated as a gap?

I guess I can imagine cases where that would be the "most correct" way to display the data, though it might be more complexity than users really want. On the other hand making a new setting for this would allow us to avoid turning connectgaps into another "boolean plus a string" enumerated attribute, as well as avoiding extra logic around its default value. And mostly people would just use the default value of this new setting. So what could this new attribute be? How about
stackgaps: ('zero' (dflt)|'gap'|'interpolate')?

[etpinard]

then there are perhaps three ways you might want to interpret gaps:

I can't think of any other scenario. Thanks for writing those down in detail 👌

How about stackgaps: ('zero' (dflt)|'gap'|'interpolate')?

To my eyes, adding a new attribute for this is a no-brainer 👍

Booleans-plus-a-string aren't great, but also any enumerated attribute that has one or multiple values that can have no effect depending on other things in that trace should be avoided when possible.

I was going to suggest fillgaps before reaching the end of your comment. But stackgaps is better as it makes it clear that this new attribute has an effect only for stacked scatter traces.

That said, could this new attribute help us alleviate some current less-than-ideal fill problems? There are many open issues about this: #1132, #1867, #113, #1205 and possibly others.

---

As an aside, this problem of mismatching x in stacked area chart appear very plotly specific. Both MATLAB and mpl assume the same independent coordinates for all their stacked area "y" arrays.

[etpinard]

Writing down some questions I have about stacked area hover:

• Should we display the "true" y datum or the stacked (i.e. cumulative) y value?
• Similarly, should we include the y datum or the stack value in the plotly_(hover|click) event data?
• Choosing the stacked value would be a first for plotly.js (I think), as we tend "true to that data"
• Displaying the y datum in the hover label could confuse some users -- especially when spikes are enabled -- as the y value read off the y-axis at the spike level would not match what's in the hover label.
• Including the stacked y value in the event data would make the event data not match the input data, which is likely to confuse several users.
• Would making the hover labels display the stacked value and include the y datum in the event data (another plotly.js first) be too confusing?

So, perhaps we could add new hoverinfo flags e.g. 'xstack', 'ystack' and making the hoverinfo default be 'x+ystack+text+name' for stacked scatter traces? These new hoverinfo keys could be used in stacked bar charts too.

[alexcjohnson]

That said, could this new attribute help us alleviate some current less-than-ideal fill problems? There are many open issues about this: #1132, #1867, #113, #1205 and possibly others.

mmm, there's some interaction between this issue and some of those - particularly #1132 and #1205 - but I think those are pretty much all implementation issues, not problems of specifying the desired behavior.

As an aside, this problem of mismatching x in stacked area chart appear very plotly specific. Both MATLAB and mpl assume the same independent coordinates for all their stacked area "y" arrays.

True, that's because both of those create all stacked lines in a single function call, so conceptually as a single object. We could in principle do the same, treating the entire stack as a single trace, but we can do better than that. I've certainly encountered plenty of situations like the population example I described, where I wanted to add up data that didn't come with matching x values, and I'd have loved it if this just worked ™️

Should we display the "true" y datum or the stacked (i.e. cumulative) y value?

hoverinfo: 'x+y+total' (or 'sum' or something). Where 'y' is the value of the trace you're focused on... I'm not quite sure whether 'total' is the partial or complete sum... I guess perhaps we want to allow both ('x+y+partial+total'?) but I'm not sure which should be the default? When you're hovering on a single point I can see wanting to know the sum of everything up to and including that point (that's where your cursor is after all... but there may also be particular subtotals you're interested in), as well as the total of everything in the stack so you can quickly see "this item is 10% of the total" (maybe we even want hoverinfo: 'percent' or something?)

Similarly, should we include the y datum or the stack value in the plotly_(hover|click) event data?

I think the y field in the event data should be the y datum but we should also include subtotal and total as separate fields.

These new hoverinfo keys could be used in stacked bar charts too.

😍

[nicolaskruchten]

I have a concern about gap ... How would that render? If I have a single data point for a given trace with gaps on either side, then it will look like an empty quadrilateral with a single dot? Otherwise it would in effect be the same as zero no?

[etpinard]

I guess perhaps we want to allow both ('x+y+partial+total'?)

I'm a big fan of this. Adding flags 'partial', 'percent' and 'total' would cover a lot of use cases 👌

I think the y field in the event data should be the y datum

I agree 100% here. Moreover, we could add partial, total and percent keys in the event data for symmetry with the hoverinfo flags.

[alexcjohnson]

I have a concern about gap ... How would that render? If I have a single data point for a given trace with gaps on either side, then it will look like an empty quadrilateral with a single dot? Otherwise it would in effect me the same as zero no?

right - an "orphan point" we've called that in the past - it doesn't make a line segment either, so if you don't show markers you won't see anything.