#django

jtiai: does pickling the .query actually achieve much in terms of performance?

Depends where session is saved.

Hey guys , how can I change django csrfmiddleware field name ?

In our case it was in db, and one query had over 2M objects causing constant copying over 30MB of data...

welp

jtiai: :o

Big object with big geometry data.

for you

i have a file about 500mb and each line has a website and i want check the status of each site and save the output of site the responds with 200k which method is better should i loop through each line using the “with open()…” or just do a normal “open()” and store the content of the file to a variable

and loop through it? One would be more memory intensive and slower I suppose… please which is better?

if you don't run many processes doing it in parallel you can get away with reading the entire file to memory

reading it line by line is probably more considerate

with open() is just a convenience for remembering to close the file.

i'm pretty sure file objects provide a convenience method for iterating over every line in the file

yeah

using the database backend for caching and now my tests fail because the cache table doesnt exist

how do i make the ./manage.py test create the cache table?

kingplusplus: how is this related to #django? may be you should try on #python

kezabelle: That's why we used query pickling since query itself is relatively simple (filtering by few fields), it's just number of objects it returns is huge.

domino14: a migration needs to create it

domino14: Or, don't use database as cache in testing.

the only table not migrated into existence is the migrations lookup table itself

./manage.py createcachetable doesnt create a mgiration

pombreda, kingplusplus: all sorts of questions are happily answered in here

hylje: thats not true, because the db cache exists in a weird way, historically, IIRC

why wouldnt that be automatic

shangxiao: sure, that one sounds a tad off topic :P

because tests only automatically run migrations and not any other magic

pombreda: perhaps, although if it's for django project…

kingplusplus: but yeah if nobody is able to help here, #python is usually active too

shangxiao: if I read kingplusplus question well, just loop through the lines with for line in .... ; how the file was opened does not matter to memory usage. How it is read does.

I mean, unless you're parallelising the task, iterating over 500MB of URLs and doing a HEAD for 200 is probably whats going to take the time, anyway

Is google spreadsheets a nosql db?

I want to replace postgres with google spreadsheets

linski: hum. Are you serious?

lol

funnily enough

there was a project that turned google spreadsheets into an orm

there's a few

to be fair google spreadsheets is a web scale managed database

linski: if anything spreadsheets are like a dirty relational database

I am sure this can be done. the question is short of a very specific use case and purpose: should is be done? :D

no

lol

I remember years ago encountering a PHP site that used Google Spreadsheets as a cached CMS

these days it seems popular again, and with node.

it _might_ be if you want to do some automated importing from that as a data source or something

google spreadsheets seems like a valid backend for a glorified static site generator

xD

all that matters is your normie content managers can use it

times are tough here

hylje: it does, but I can't see how it'd be a good way of managing large chunks of content

times will be tougher if you replace postgres with spreadsheets xD

so Google sheet can be seen a multi-user, cloud-based, scalable, schema-less database.The same way emacs does the same, but eschewing the complexity of storing things in rows/columns together and for single users only

I mean ... the cells are tiny. they'd need to be at least 3 times bigger

you can build a form to update/add/edit your spreadsheet data

oh can you?

yep

TIL.

yeah some people use a public gspread form to do polls and questionnaires

yeah I've seen them, but never clicked on them ;)

heh

hey FunkyBob I was reading up on the GCBV earlier, I think UpdateView might not be the way to go as I want to 'create' notes and not 'update' tehm

After googling and stuffs, I saw a post where someone said it's possible to use detailview and add a form in the context (which is my current status) and point it towards a seperate formview

linski: so I guess the comments here provide some hints: this is not a nosql DB, but a spreadsheet tool that can store rows and columns in a limited way. This hardly can be seen as a replacement for Postgres. The same way a hoverboard shares some properties with a car, it cannot replace a car. And I do not know of any backend for Django to use that.

FunkyBob has something to say about the label 'nosql'

:D

Thanks for help :)

ArangoDB is pretty neat for no-sql data... :)

you meant: no? sql!

Actually arangodb query language is more sql than sql itself.. ;)

i hear that a lot

i must try it

It's pretty neat. We use arangodb to store some documents, mainly checklist types.

hi, I have lots of doubts and fears when editing model fields and running migrations having had lots of problems with it.. Can anyone tell me what is a safe way to modify a decimalfield previously set with null= True, deleting this option? I tried, ran makemigrations and returned error..

rufus: So what's the error..?

probably there was null data? ;)

hey guys, http://dpaste.com/0DB06BQ This is my current detailview. I'm trying to accomplish having a form to save NEW 'snippets/notes' in the same page as the details page. Right now the form is appearing as I saved it in the context, but it's not saving when I click submit

anyone can give a hand?

rufus: can you use dpaste.de for details?

kezabelle: yes there were some

rufus: well then you'd need to fix that before you could migrate

my code is kinda from the docs.

usually with a data migration

jtiai: many errors, last line shows: unable to ALTER TABLE as there are pending trigger events

I'm guessing it's because I didn't have some kinda obj.save?

kezabelle: yes how to fix this? I had options and chose to have manually filled the null values..

PickAndMix: ... now I know why I never dare to wander to land of GCBVs... :)

jtiai: heh

jtiai: T.T

and a first-timer like me wanders in cluelessly

That's why I like FBVs, it's much harder to get lost.

jtiai: are you able to achieve all these with FBV?

jtiai: a detail view and a create view together?

I'll probably give that a try on another app.

I can do whatever I want to since it would be full control for me.

Of course I've used plain CBVs rather successfully, it's just generics that gives me a big creeps.

T.T

jtiai: haha

jtiai: I guess if FBV are working great for you, there's no reason to change

PickAndMix: http://musings.tinbrain.net/blog/2015/jun/10/up... that's what you were asking for?

pombreda: here is the error generated https://dpaste.de/SVGc

jtiai: I think I'm getting the result I want, right now I just need to find a way to link up the "task.id" to my notes model.

lol I'm so bad at django

rufus: seeing Python exceptions stack traces in Italian is a first for me :P

pombreda: yeah last line means: unable to ALTER TABLE as there are pending trigger events

hi all. just a quick question: I'm about to add a bunch of models to an application with multiple django apps. The UI will be closely related to an existing app, but the functionalliity will be a different one. When shoud I create a new django app?

sebastian-w: Good rule of thumb: app does one thing, and one thing only - and does it well.

jtiai: thanks!

kezabelle: could I just delete fields (database is still on test) migrate and then rebuild the new model with fields deleting Null = True? Will this work?

rufus: Depends do you have to update production database(s) that may have nulls.

rufus: this is a DB locking issue. Have you tried to stop and restart your DB server?

at least thei will not hurt

*this