#appengine

plantedseed: If your Player models are at the top level, each player can have one update per second.

But the other side of that is that any update made without an ancestor takes time to propagate throughout the system.

Lenolium: I get the error BadRequestError: queries inside transactions must have ancestors

are transactions pointless without ancestors? i dont get it

don't use queries in transactions

do only key based lookups

build your own indexes to keep track of whatever you want

if necessary run cron jobs that will do a query and then save that query's result to some entities that you can fetch by key in a transaction

but in most cases you don't need the query job and can design the data model in a way that you can just chain key based lookups

i dont get it. I signup, that means checking if the username and email is taken, if not fine => insert else return error to the user

yeah, for checking that I suggest lock entities for username & email

so an entity UsernameLock, that would use the username as key, and also have a nother property which contains the user's key

then you can fetch that userlock entity by key (which is the username) to check in a consistent way whether the username has been takend or not

so I need to query and insert in a transaction s no other player comes between and also checks, sees that it is fine at the same time and then i get two inserts of the same username

so i create a parent?

no, why do you need to query? don't query

how can i not query?

what is that you want to do? check for if username is in use?

yes and email

I suggest lock entities for username & email

so an entity UsernameLock, that would use the username as key, and also have a nother property which contains the user's key

then you can fetch that userlock entity by key (which is the username) to check in a consistent way whether the username has been takend or not

Problem 1: username and email must be unique Problem 2: Save progress, email, etc. Saving means checking what level the user is at in the daatabase and only store if higher. Then I have to query right?

Strom: but is that a real lock? cant someone other user do the same at the same time and both see that the username is free and insert?

that's what transactions are efor

you do it all inside a single transaction

1) check for UsernameLock by key (which is the username) -- if it doesn't exist, then the username hasn't been taken and you can proceed

2) creat a new user with auto-generated numeric id

3) save a UsernameLock entity, with using the username as the key, and also putting the auto-generated numeric id as a property

if two users start trying to do it at the same time, then the transaction will succeed only for one of them

the other user will get an error -- the transaction will first fail, and then the automatic retry will see that the UsernameLock is in place and the username has been taken

-- and for your saving / user level check, you can do that in a transaction without queries as well

ok thanks. so i need 2 parents? UserNameLock and EmailLock?

you can fetch the entity by key that contains the level information

parents? no don't use the ancestor system at all really

will this means that "If your Player models are at the top level, each player can have one update per second." will not be true anymore?

it will

it will be ture, to be clear

*true

OK, so if I keep updates two one particular entity with a minimum of 10 seconds then i should never run into problems with transactions failing?

two->to

Strom: OK i am not sure still. UsernameLock would be a class UsernameLock(ndb.model) and there would be one for each username? why not make the username the key for the Player model then? WOuld the Player and the UsernameLock reference each other somehow?

I need a model with 2 keys? is that possible? Or I need the Player model/entity type with username as Key and then i need another model with email as a Key?

When will memcache be available on app engine flexible?

Strom

Can someone please helpt me with those questions+

plantedseed: you can use cross-entity group transactions ("xg" in ndb )

this allows you to use a transaction across more entities groups that do not share a common ancestor

xg-transactions are very useful for rare situations that would otherwise cause you to change your datamodel in weird ways

so you just get start a xg-transactions, get the User entity, get the Email entity and then do your thing

ty. so user and email will be different entities, not properties?

right now i have them in the same entity/model

Yes think you want to have them different entities

and of course they need to refer to each other through a property

and how does this affect the possibility to query often? i mean with a million users there will be quiet a lot of signups and logins

what do you need to query for?

the user needs to be able to update his progress every 5-10 seconds (that wont happen very often but in the worst case)

not query but use the database

as in "check value , write"

datastore performance will not be your issue :)

as long as you write to different entity gorups

datastore has excellent 'horizontal' scaling.. that is you should model your data so that each user's data is in its own entity group

ok, so how many entities/models do i need to describe a Player which has the properties: username and email which are unique and then level, country and some other misc fields

wont a cross-group transaction affect the write contention for all entities?

only those in the transaction

the entity groups used in the transaction

I can't make your app for you.. i would just start making it to get experience

but if i get 1 million users that 1 million entity groups right? the limit is at 25...

yes

you can refer to max 25 entity groups in an xg transaction.. but you only need to access 2

your User/Player entity and the "Email" entity

Player entity has the username as its keyname, Email entity has the email address as keyname..

and Email entity of course needs a property that stores the associated username and viceversa

I don't think I can explain it any better, sorry

bbl going for coffee

but each entity is in its own group isnt it? if it does not hve an ancestor?

if they have ancestors then all the sudden you can only make 1 update per second to the dataset

you are saying the XG transaction applies only to the WRITE part? looking up by KEY for all entities is not included in the 25 entity groups?

it is included in the 25, if it's inside the transaction

OK, im still confused by all this.

User 1 signs up as "user1" and "[email protected]";, is it not possible to prevent anyone else from signing up with the same username at the same time without a huge penalty that will not work?

it is possible

I gave you the transaction actins step-by-step yesterday

yes thanks but i dont really get it.

what don't you get?

why would i have a separate UsernameLock, why not just have the username as key in the real User/player model? And then I need a EmailLock for the email?

For one user, what do I need and how would the transactions look? 2 or 3 models? Main model/entity with data, 2 lock entities?

1) you could use the username as a key for the actual User entity, but then it would be difficult to let users change their username

and those entities, there will be more than 25 users then there are more than 25 XG and I cannot do a transaction over ALL users, then it falls...

2) you would still need a separate entity for other things you want to ensure are unique as the e-mail

Usernames will not change

email can be changed

why do you need to do a transaction over all the users?

(it's possible, but not performant)

how else would i ensure noone tries the same username?

did you look at the link I posted?

just to be clear, let me paste it out one more time

1) check for UsernameLock by key (which is the username) -- if it doesn't exist, then the username hasn't been taken and you can proceed

isnt that what the transaction is for: 1. Lock database. 2. Check if username exists. 3. Insert if not existing.

so noone can come in between point 2 and 3 and think hey username does not exist lets insert and then you get 2 identical usernames