#openspending

Hey pudo :).

Quick question. I'm looking to work on a core viz data model, not tied to any particular backend, but that would provide a consistent API for views. I started sketching ideas here: https://docs.google.com/document/d/16bDDxYfUD1C... to flesh out

and now I just opened up spend.ui to have a looksee

it looks like the "data model" for the viz is quite tied to the rest of the app... e.g.: https://github.com/spendb/spendb.ui/blob/master...

a goal for me right now is to bring together some really basic data model for viz based on measures and dimensions, but not tied to FDP or any particular backend, and to see how I can bring the work of spendb, particularly viz, into openspending.

the question is: what other viz aspects do i need to look at - e.g.: also babbage.ui?

it looks like most of the work is there in those dataset/* scripts, but if you have any other pointers, feel free

pwalsh: hey man, just got online

so yeah, all the viz stuff in spendb is in babbage.ui

it's somewhat tied to babbage, but really just to the notion of dimensions, attributes and measures - which I guess FDP now also has

(in fact, FDP and babbage have gotten really close)

this could be a good place to get started with the viz stuff: https://github.com/spendb/babbage.ui/blob/gh-pa...

pudo that is good :)

that they are getting close

it's the sidebar code which constructs queries

yes, it's awesome

i think we still need work, of course. but i can see we'll get to a good place with FDP soon

I'm keen to see how you guys will tackle hierarchies and what the final descriptors for classifications will look like

duo that is really the difficult thing right now as far as I see :)

yeah

* that is the...

I spent a lot of time discussing hierarchies with aristaran, he's using mondrian for his spending db

they do it similar to cubes

but I just struggle to think of how to hide the complexity from users

I'm personally less focussed on that right now, Rufus and Dan are more on that (spec). Complexity: yes. I have no ready solution for that at this stage. But looking at it again

Perhaps it'll require introducing a third stage in the dimension modelling: select all attributes in a dimension, then all hierarchies, then really spec it out

awkward

but it would be really helpful with vis

i.e. I need to hack it in via https://github.com/okfde/offenerhaushalt.de/blo... right now

i'm quite keen to work on a data model thing though for viz now... (as noted above). Very similar to what you have in spendb as I see. First I'll work towards some general ideal, and then start to look at a) how it looks with Dan's current work, and b) how it looks with spendb viz layer

let me know if I can help in any way

hoping to deploy babbage in a non-SpenDB context at work soon, so it'll need to go a bit more generic anyway

I'd like that (help). First I'll start with a basic doc of the requirements (quite short), and we can riff on that. I'm actively working on this today and tomorrow. I'll update here.

great.

pudo now that you are down that road, how do you feel about babbage compared to cubes? Are you planning to keep working on babbage?

I feel quite good about it, to be honest :) it has very high test coverage and very limited functionality

the exact opposite of cubes in that sense, I'm afraid

cubes tries to do N backends (mongo, mixalytics, ...)

and then has weird bugs like not escaping column names (i.e. 'time' as column queries the function instead)

the biggest missing function in babbage is joins/star schema

that's doable, but I wanted to see a real need for it first

classification alignment would be such a need

pwalsh: just looking at your diagram -

one thing I tried and that really didn't work was standardizing vis state

each type ends up having different numbers of axes

so you can standardize filters, but not breakdowns

what exactly do you mean by standardising viz state?

the last part of your diagram

says you want to standardise around API state

which I assume to mean: "show me viz type X with the following query as an input"

yep.

so that works for filters

but not so well for the dimensional drilldowns

yep. i see.

i don't want to start working on a big spec for this now :). just the simplest possible api for measures and dimensions on spend data. i guess right now, until we can power stuff like the spendb viz from it, etc.... state is a secondary concern.

so let's take it up later

ok

about babbage - ok. good overview, thanks. I'm mostly wondering about querying/efficiency/star schema but it is theoretical at this point. congrats on the test coverage btw :)

as a quasi-random side note, I'd love to see a set of two functions that take any babbage query and generate a the same SHA1 both in JS and Python land

that way, we could use static data

interesting ;)

(that's what cubepress was initially intended to do... we're doing a few commercial projects with OKF-DE soon, I'll try and get Matt to implement a version of it ...)

pwalsh: still around?

yes man

ha. I've been seeing your tickets on github come in, and realize that I've completely lost any model of how all of these pieces fit together. What's the best reference on that, atm?

the data loader, cli, and dan's frontend stuff

hi. sorry, got distracted. I'm at a HaSadna meet up...

looks like my nice image for that has been moved from the community site, but this is the best overview I'd say: https://github.com/openspending/dotorg/blob/791...