#django

the more magic way? probably just render the template on the server, and let IntercoolerJS replace it :o)

Even plain JS can do that :D

jtiai: yes, if you feel like writing a bunch of JS :o)

jtiai: use which frameworks?

I use sass and jquery write for my front code

so we are refactoring a big app into an extendable set of apps, a la pinax project

the issue now is how to customize and override behavior and fields on models

what's your experience using django on this issue?

There isn't a clean way to replace fields on arbitrary models.

But you can add your own models that link to other app's models.

abstract models containing the non-replaceable ones, concrete models for replaceable ones, possibly using swappable models if necessary, or onetoones if appropriate

mattmcc, yep, but in that case I couldn't customize behavior at the model level; let's say I want a custom manager in an entity for a project. I'd need to swap the default one

kezabelle: But the app(s) in question have to be written to support that.

mattmcc: sure sure

interesting approach kezabelle, what do you mean by swappable?

Honestly if you need to extensively modify models in third party apps, fork them.

mattmcc: yeah but I'm assuming that if they're refactoring it, they control the apps

that's correct, we control the apps, we just want to make them "pluggable"

Ah, yeah, then you can certainly design them that way.

its a bunch more work, obviously ;)

I like the concept of swappable models, it's a fancy class dictionary; I was thinking along those lines

one day, swappable models may be codified as public API in Django proper (outside of auth etc), until then ... swapper.

Yeah, I'd just keep in mind that swapping the model after its table exists is nontrivial.

fair warning, I've not used it in anger/production, but it seemed to work when I played with it.

Like, changing AUTH_USER_MODEL after you're live is not something you'd want to do.

we're working on this as if it was a greenfield project, later on we will migrate using good ol sql

I thought about using single table inheritance for this matter, but swappable models are just better

we don't need for the time being polymorphic models, we just need to swap them :)

It is possible to leverage lots of the swappable stuff. I did it for jango-boardinghouse.

damn auto carrot!

:)

did you use swapper?

or did you wade into the internals yourself?

I think I did look at it at some point.

There was a fair bit of internal-poking going on for that app in general...

well that's no use. I want someone I trust to have used it already so I can feel OK about mentioning it ;)

In reality, I don't think I found a compelling reason to use it over just making the model swappable.

(And then having a `get_schema_model()` function that looks up the model from settings)

but dependency magic is magic that's not my problem ;)

(allegedly)

IIRC, it was literally one Meta option, and one 8-line function (that is mostly just try: except ...)

I assume being a superuser isn't enough, but one has to add them to ADMINS setting to get error emails?

Right, that's the sole point of the setting.

A todo for tomorrow then. Ty.

That said, I would submit that the use of AdminEmailHandler doesn't scale to any nontrivial production environment.

It doesn't do sampling or rate limiting.

Ill have to look into it tomorrow. Right now I'm soley relying on apache's error long in production.

Very trivial ;)

There are a variety of options. I prefer installing a local instance of Sentry.

Sentry link under 1.8 deployment checklist is a 404. Oops.

i'm looking for some way to de-duplicate scheduling celery task (i'm doing those in on_commit), got any ideas?

basically i don't want to think whether i already scheduled reindex task or if i still need to do this

AtomicSpark: Which link are you referring to?

i just want to fire it at all places appropriate

nazarewk: you could use a distributed lock with redis

nazarewk: What if, while it's running, a change is made that actually does require a reindex?

I've done it recently.. and while there are some gotchyas when implementing it correctly, the naive implementation can be ok-ish

mattmcc: it will still fire AFTER the transactions

Or is there a time interval (seconds, minutes..) where the index being out of date is acceptable?

A lock would normally block queueing additional reindexes, rather than delay/serialize them.

Sentry link is 404.

mattmcc: i am aware that somewhere else there can be a change that requires reindexing, i just want to prevent scheduling bazillion tasks for the same thing for example in a for loop

yes, that is correct.. but you can readd yourself to the queue if the lock is locked

in a single transaction

AtomicSpark: Huh. Didn't know the docs linked there. That's a docu bug, then.

ahh but that won't prevent adding a new one to the scheduler

nazarewk that sounds like bad design, but if you are up to it, I'd wrap the thing into a transaction. you create a context for it, once the context is consumed, you schedule the task

there's also the on_commit signal, no?

yeah, but on_commit is not preventing duplicates

it would be ideal if it did

no, it'd only prevent multiple per request

nazarewk: If it were me, what I'd do is write my Celery task such that whenever it gets called, it looks for the latest reindex time. If that time is 'too long ago', run. Else, bail.

and check again when it's actually scheduled that it's not already running :D

Right.

mattmcc: i already incorporated non-blocking lock into the task

i am not sure if celery provides efficient way to check whether task is already in queue

according to http://stackoverflow.com/a/34290708/10583 there is not an efficient way

but the recommendation there is to use a counter

have a separate queue which only allows one task/worker at a time :o)

and be super careful not to schedule time sensitive tasks there hehe

nazarewk: Celery doesn't have a singleton enforcer, no.

jarshwah: indeed :)

Eh, that wasn't really your question.

But still, I would use my own mechanism to determine if a task should be queued again vs 'is already in the queue' because that would seem to depend on business rules.

basically i have 3 models that should fire reindex on save

they could all get saved in a single transaction

instead of Redis, if you are using just one machine a file lock should be enough

can always just use a sentinel lock, whether that be a file or a redis key or whatever

that's what http://stackoverflow.com/a/34290708/10583 suggests.. did you read it nazarewk?

but you'd have to lock eagerly in-band, and then have the celery task remove the lock on successful completion

well, going this way in-memory lock should be enough

jarshwah: skimmed over it, gonna read more in-depth

in memory where? you can't remove the lock from the worker unless it's shared via memcached or redis or whatever

if you lock inside a Django process, and need to unlock inside a worker on successful complete, it needs to be accessible to both via a shared mechanism, which LocMem would not be (for example)

If one uses sentry for a small host, should I go with the free account or self host? The documentation kind of warns against the latter in an almost sales man way.

i don't really care about multipler workers scheduling tasks, i just don't want a single worker to schedule tasks multiple times in a single context

then be able to attach the context to the request, be it a threadlocal or whatever, and make a transaction out of it

are they? :)

the most common sense solution, imho, is mattmcc, mixed with kezabelle suggestion of a shared lock

"Don't use threadlocals, like we do everywhere" -Django

Most errors should be caught at dev level anyways. The only actual errors I've gotten so far were from Python version mismatch or Apache permission issues. Both of which just means the site is down.

mattmcc: not *everywhere* ... sometimes there's just a global ;)