#ckan

/

      • denisz joined the channel
      • kirembu has quit
      • kirembu joined the channel
      • nigelb has quit
      • nigelb joined the channel
      • denisz has quit
      • tomkralidis joined the channel
      • tomkralidis has quit
      • amercader joined the channel
      • mlechner joined the channel
      • batje joined the channel
      • batje has quit
      • batje joined the channel
      • amercader joined the channel
      • batje has quit
      • batje joined the channel
      • dragon joined the channel
      • dragon is now known as Guest74261
      • batje has quit
      • batje joined the channel
      • Guest74261 is now known as DragonDave
      • rossjones
        seanh_ mostly in templates, and serving the files from nginx.
      • seanh_
        I've just used template-overriding to insert the tags, and fetching the files from jquery's cdn so far
      • rossjones
        When the workaround is dramatically less effort than the 'standard' approach, I think there's probably something very wrong.
      • seanh_
        Yep. I'm not sure how much fanstatic itself is to blame, and how much it's the way it's been integrated into CKAN
      • But it was all done to reduce page-load times I believe, seems to have added a lot of difficulty in the process
      • I wouldn't mind a more indirect process for serving static files so much if it were properly documented and it worked, but I spent half a day failing to get one js lib to work that took two minutes to do without fanstatic
      • nigelb
        If we can fix it in a way that static files are served by the webserver and wsgi, I'll be *very* happy :)
      • rossjones
        I think a few are unhappy with serving content through wsgi as well, I think removing it will be a complete nightmare though
      • nigelb
        *webserver and *not*
      • nickstenn
        rossjones, nigelb, seanh_: there's just no reason at all that assets should be served through wsgi. the "page load times" argument is solved by almost everyone else by precompiling assets
      • seanh_
        Right, the whole point of bringing in fanstatic (as I recall) was that it could minify and bundle CS and JSS files, and also track dependencies so we could easily have each page load all the files it needs and only those files. If we could get the same benefits in a less invasive way that would be great
      • nickstenn
        serving file content is also not a fabulous idea, although I imagine there are authorisation issues, in which case it would probably be a case of setting up X-Accel-Redirect and friends (http://wiki.nginx.org/X-accel)
      • seanh_
        Fanstatic also supports running compilers for less or whatever though I'm not even sure we use those features
      • nickstenn
        seanh_: absolutely, but those should all run out of band and create files on disk that can be served by a conventional webserver
      • this is how rails does it, how django does it, and how basically every sane asset pipeline anywhere does it
      • rossjones
        nickstenn problem now is how to fix it, if people are writing extensions that make use of it :(
      • nickstenn
        s/how django does it/how many django extensions do it/
      • rossjones: presumably you could keep one single fanstatic tag for extensions, and deprecate it?
      • rossjones
        I guess so, but deprecating doesn't really mean anything in ckan, or rather it just means "we don't use this anymore".
      • Doesn't mean it'll ever get removed.
      • nickstenn
        ARGH
      • rossjones
        Or that ppl will stop using it (see Genshi)
      • nickstenn
        WELL LET'S FIX THAT THEN
      • (frustration not directed at you)
      • rossjones
        It's my one biggest concern about moving to gunicorn is wasting workers on fileserving.
      • nickstenn
        rossjones: I don't think it'll be any more of an issue than with apache with an evented worker
      • with sync workers it would be a very bad idea, yes
      • rossjones
        nickstenn probably right.
      • There should be a list from core team "Things I would remove, break, refactor if I had time". I guess that's the idea of http://github.com/ckan/ideas-and-roadmap/
      • nickstenn
        rossjones: so there's no systematic deprecation tagging and warning setup in CKAN at the moment?
      • rossjones
        I think some things give warnings.
      • DGU still uses Genshi, so I suspect they'll have a LOT of work to remove that, for no immediate gain (for them). Apart from jinja being faster and cleaner.
      • nickstenn
        at some point, someone decided that the trade-off between up-front effort removing genshi and efficiency gains from removing it were worthwhile
      • nigelb
        nickstenn: the biggest argument for fanstatic right now is extensions
      • it deals with getting the static files from extensions and doing the right thing.
      • nickstenn
        either a) that is still true or b) it is not
      • nigelb
        the minimization and other stuff can be easily fixed.
      • nickstenn
        we can't expect to get anywhere by having an existential crisis halfway through about whether the DGU team will ever upgrade
      • if they need to write a CKAN extension to support their legacy templates, then let them
      • nigelb
        I see your comment about django extensions
      • nigelb looks
      • nickstenn
        nigelb: have a look at foundation
      • anyway, the core team's responsibility is to improve CKAN as a product
      • there is a place for concerns around upgrade path and backwards compatibility, but if those concerns are sucking most of the forwards momentum out of product development, they need to be relaxed
      • </rant>
      • nigelb
        django things are great for django.
      • I'm looking for something that works outside of django as well.
      • rossjones
        good rant
      • nigelb There are other non-python, non-integrated solutions to the problem.
      • nigelb
      • seems like a good place to start
      • batje has quit
      • batje joined the channel
      • seanh_
        nickstenn: rossjones There is actually a proper deprecation policy, there's an @deprecated decorator that prints out a warning, and the release notes for every release have a deprecations section, there's something in the contrib guide about it
      • We have generally been very conservative about not breaking things for clients, true. I think it's partially a backlash against the 1.* days when every CKAN release broke very site and extension in all sorts of ways.
      • nigelb
        I'm okay with breaking things if we write a proper upgrade guide and warn the list in advance.
      • seanh_
        Things like having a stabler API for extensions with a deprecation policy (and that includes things like plugin interfaces, template helper functions .. not just the actual API) have been good I think (also for our own sanity, we maintain a bunch of CKAN extensions)
      • But I'm generally ok with breaking things in order to remove total crap from CKAN, Genshi would be an example
      • nickstenn
        seanh_: Right, but it's important to remember that a responsible change policy is one that has strict guidelines about *when* you can change things in a way that breaks other people's code (i.e. only on major versions). It's important that we don't take that to some kind of bizarre extreme that says it's *never* ok to break others' code.
      • seanh_
        Only on major versions - this is clearly documented
      • There's still a case-by-case discussion to be had on specific things like we want to remove Fanstatic or Genshi, there's still a trade-off between simplifying/improving CKAN versus breakage that has to be an ongoing discussion
      • I'm more on the pro breakage side
      • nickstenn
        yay :)
      • seanh_
        More specifically, there are certain things in CKAN that are "good", that have been designed and implemented fairly well and are well-documented, and I wouldn't want to break those without good reason. The API, plugin interfaces and toolkit, template helper functions, etc etc. Basically anything that' s well documented on docs.ckan.org
      • Other things in CKAN suck, including Fanstatic (the docs on how to use CKAN's Fanstatic integration are not very good), Genshi is essentially a hidden feature, the templates and their blocks are very complex and not documented ...
      • nigelb
        *cough* revisions
      • seanh_
        Technically extensions and themes may be relying on these things, but they're things that were just hacked in to CKAN because we needed them at the time without a lot of thought. I'd break them in order to get them to something good that's actually documented and designed to be built on.
      • shellac joined the channel
      • rossjones
        so things to go are fanstatic, genshi, revision, resource_groups, using solr as a database ... anything else?
      • nigelb
        datapusher
      • (I'm... not a fan of how it's done)
      • rossjones
        ah the queuey thing.
      • amln
        there are only 89 email addresses on ckan-announce
      • rossjones
        amln how many on ckan-dev?
      • nigelb
        ckan-announce is for people who don't want the noise on ckan-dev
      • I'm not sad about that few.
      • amln
        rossjones: 413 on ckan-dev
      • you'd think you might want to filter annoucements…
      • don't most people sieve/procmail announce lists differently?
      • (or whatever the gmail equiv is)
      • DragonDave has quit
      • DragonDave joined the channel
      • is anyone aware of ckan instances with two frontends?
      • amercader
        why can people post to ckan-announce? I though only the core team could do it
      • oh, and I'm and admin and need to approve my own post? mmm
      • amln
        amercader: people could post. i changed it so everyone is held.
      • remember, if i spoof your from:/sender: line in a list mail, i can post as you.
      • and it will just get through.
      • MM3 has some improvements in this area.
      • amercader
        amln: but it will need to be approved anyway right?
      • amln
        now, yes ;)
      • batje has quit
      • also saves the issue of sending an offlist reply to the list…
      • s/saves/sometimes protects from/
      • seanh_
        rossjones: I imagine there's a few more things like that to go as well :)
      • But yes
      • I'm not sure about using solr as a database - isn't that done because it's so much faster?
      • maxious
        speaking of ckan contracts with specific technology requirements, I fixed some stuff in ckanext-drupal7 https://github.com/okfn/ckanext-drupal7/pull/1
      • rossjones
        seanh_ yes, don't get me wrong, I am grateful it got us all past the perf hump, but maybe it'd be unnecessary if the db model was simpler.
      • maxious is someone *making* you use Drupal? That's not nice :(
      • maxious
        at least it's not wordpress :P
      • batje joined the channel
      • amercader
        maxious: that's great! tbh you are probably the current world expert in ckanext-drupal7, do you want us to give you admin rights so you can mantain it? I fear that otherwise your PR will get ignored
      • maxious
        amercader: that would be great :)
      • tomkralidis joined the channel
      • amercader
        maxious: can you check I didn't mess up? I also moved the repo to the ckan org btw
      • redirects should be in place
      • maxious
        looks good
      • amercader
        maxious: awesome, thanks for helping take care of it
      • maxious
        no problem, should get a chance to do some more work on it soon.
      • tomkralidis has quit
      • redShadow joined the channel
      • mlechner has quit
      • wardi
        ooh that sounds interesting
      • denisz joined the channel
      • fanstatic is a nightmare, fanstatic_resources.py is a horror-show
      • amln stills reads fanstatic as fantastic
      • nigelb: fanstatic does the wrong thing. all static file paths are merged with the root application paths
      • seanh_
        maxious: I didn't even know ckanext-drupal existed! Great to see someone maintaining it, ckan-drupal integration is something we get asked for a lot (e.g. when delivering ckan trainings). Also ckan-wordpress integration. It's also the whole reason why dkan exists as far as I can tell, but instead of just integrating ckan and drupal they're reimplementing ckan in drupal
      • wardi
        rossjones: yes, i've used the ideas repo for this sort of thing, along with the 'technical debt' (brown) tag
      • rossjones
        good choice of colour :)
      • kalxas joined the channel
      • batje has quit
      • denisz has quit
      • realitygaps has quit
      • realitygaps joined the channel
      • realitygaps has quit