plantedseed: If your Player models are at the top level, each player can have one update per second.
But the other side of that is that any update made without an ancestor takes time to propagate throughout the system.
kingarmadillo joined the channel
buffal0 joined the channel
buffal0 has quit
buffal0 joined the channel
buffffal00 joined the channel
buffal0 has quit
buffffal00 has quit
buffal0 joined the channel
plantedseed
Lenolium: I get the error BadRequestError: queries inside transactions must have ancestors
are transactions pointless without ancestors? i dont get it
Strom
don't use queries in transactions
do only key based lookups
build your own indexes to keep track of whatever you want
if necessary run cron jobs that will do a query and then save that query's result to some entities that you can fetch by key in a transaction
but in most cases you don't need the query job and can design the data model in a way that you can just chain key based lookups
plantedseed
i dont get it. I signup, that means checking if the username and email is taken, if not fine => insert else return error to the user
Strom
yeah, for checking that I suggest lock entities for username & email
so an entity UsernameLock, that would use the username as key, and also have a nother property which contains the user's key
then you can fetch that userlock entity by key (which is the username) to check in a consistent way whether the username has been takend or not
plantedseed
so I need to query and insert in a transaction s no other player comes between and also checks, sees that it is fine at the same time and then i get two inserts of the same username
so i create a parent?
Strom
no, why do you need to query? don't query
plantedseed
how can i not query?
Strom
what is that you want to do? check for if username is in use?
plantedseed
yes and email
Strom
I suggest lock entities for username & email
so an entity UsernameLock, that would use the username as key, and also have a nother property which contains the user's key
then you can fetch that userlock entity by key (which is the username) to check in a consistent way whether the username has been takend or not
plantedseed
Problem 1: username and email must be unique Problem 2: Save progress, email, etc. Saving means checking what level the user is at in the daatabase and only store if higher. Then I have to query right?
Strom: but is that a real lock? cant someone other user do the same at the same time and both see that the username is free and insert?
buffal0 has quit
Strom
that's what transactions are efor
you do it all inside a single transaction
1) check for UsernameLock by key (which is the username) -- if it doesn't exist, then the username hasn't been taken and you can proceed
2) creat a new user with auto-generated numeric id
3) save a UsernameLock entity, with using the username as the key, and also putting the auto-generated numeric id as a property
if two users start trying to do it at the same time, then the transaction will succeed only for one of them
the other user will get an error -- the transaction will first fail, and then the automatic retry will see that the UsernameLock is in place and the username has been taken
-- and for your saving / user level check, you can do that in a transaction without queries as well
plantedseed
ok thanks. so i need 2 parents? UserNameLock and EmailLock?
Strom
you can fetch the entity by key that contains the level information
parents? no don't use the ancestor system at all really
plantedseed
will this means that "If your Player models are at the top level, each player can have one update per second." will not be true anymore?
Strom
it will
it will be ture, to be clear
*true
plantedseed
OK, so if I keep updates two one particular entity with a minimum of 10 seconds then i should never run into problems with transactions failing?
two->to
Strom: OK i am not sure still. UsernameLock would be a class UsernameLock(ndb.model) and there would be one for each username? why not make the username the key for the Player model then? WOuld the Player and the UsernameLock reference each other somehow?
I need a model with 2 keys? is that possible? Or I need the Player model/entity type with username as Key and then i need another model with email as a Key?
plantedseed has quit
kloff joined the channel
kingarmadillo joined the channel
kingarmadillo has quit
kloff has quit
plantedseed joined the channel
telex has quit
telex joined the channel
[1]slugmandrew joined the channel
slugmandrew has quit
[1]slugmandrew is now known as slugmandrew
kingarmadillo joined the channel
kingarmadillo has quit
plantedseed has quit
kingarmadillo joined the channel
kingarmadillo has quit
kingarmadillo joined the channel
aatish910 joined the channel
aatish910
When will memcache be available on app engine flexible?
plantedseed: you can use cross-entity group transactions ("xg" in ndb )
this allows you to use a transaction across more entities groups that do not share a common ancestor
xg-transactions are very useful for rare situations that would otherwise cause you to change your datamodel in weird ways
so you just get start a xg-transactions, get the User entity, get the Email entity and then do your thing
plantedseed
ty. so user and email will be different entities, not properties?
right now i have them in the same entity/model
troberti
Yes think you want to have them different entities
and of course they need to refer to each other through a property
plantedseed
and how does this affect the possibility to query often? i mean with a million users there will be quiet a lot of signups and logins
troberti
what do you need to query for?
plantedseed
the user needs to be able to update his progress every 5-10 seconds (that wont happen very often but in the worst case)
not query but use the database
as in "check value , write"
troberti
datastore performance will not be your issue :)
as long as you write to different entity gorups
buffal0 joined the channel
datastore has excellent 'horizontal' scaling.. that is you should model your data so that each user's data is in its own entity group
buffffal00 joined the channel
plantedseed
ok, so how many entities/models do i need to describe a Player which has the properties: username and email which are unique and then level, country and some other misc fields
wont a cross-group transaction affect the write contention for all entities?
troberti
only those in the transaction
the entity groups used in the transaction
I can't make your app for you.. i would just start making it to get experience
plantedseed
but if i get 1 million users that 1 million entity groups right? the limit is at 25...
troberti
yes
you can refer to max 25 entity groups in an xg transaction.. but you only need to access 2
your User/Player entity and the "Email" entity
Player entity has the username as its keyname, Email entity has the email address as keyname..
and Email entity of course needs a property that stores the associated username and viceversa
I don't think I can explain it any better, sorry
bbl going for coffee
plantedseed
but each entity is in its own group isnt it? if it does not hve an ancestor?
if they have ancestors then all the sudden you can only make 1 update per second to the dataset
greg_f joined the channel
you are saying the XG transaction applies only to the WRITE part? looking up by KEY for all entities is not included in the 25 entity groups?
buffffal00 has quit
Strom
it is included in the 25, if it's inside the transaction
smw has quit
buffal0 joined the channel
plantedseed
OK, im still confused by all this.
kingarmadillo joined the channel
User 1 signs up as "user1" and "[email protected]";, is it not possible to prevent anyone else from signing up with the same username at the same time without a huge penalty that will not work?
smw joined the channel
Strom
it is possible
I gave you the transaction actins step-by-step yesterday
why would i have a separate UsernameLock, why not just have the username as key in the real User/player model? And then I need a EmailLock for the email?
buffal0 has quit
For one user, what do I need and how would the transactions look? 2 or 3 models? Main model/entity with data, 2 lock entities?
Strom
1) you could use the username as a key for the actual User entity, but then it would be difficult to let users change their username
plantedseed
and those entities, there will be more than 25 users then there are more than 25 XG and I cannot do a transaction over ALL users, then it falls...
Strom
2) you would still need a separate entity for other things you want to ensure are unique as the e-mail
plantedseed
Usernames will not change
email can be changed
Strom
why do you need to do a transaction over all the users?
(it's possible, but not performant)
plantedseed
how else would i ensure noone tries the same username?
Strom
did you look at the link I posted?
just to be clear, let me paste it out one more time
1) check for UsernameLock by key (which is the username) -- if it doesn't exist, then the username hasn't been taken and you can proceed
plantedseed
isnt that what the transaction is for: 1. Lock database. 2. Check if username exists. 3. Insert if not existing.
so noone can come in between point 2 and 3 and think hey username does not exist lets insert and then you get 2 identical usernames