#twisted

Hey guys, I'm running a Flask webapp using twisted, and I now want to daemonize the process. So I'm trying to use the daemonize PyPI package, which does the common things (forks, sets fd 0/1/2 to /dev/null, etc), however my reactor crashes when daemonized, it tries to close an invalid file descriptor which is just some python file that was read earlier.

The traceback I get is this: https://gist.github.com/anonymous/be4c561fac304...

The whole thing is baffling, the file descriptor was it reading some random .py file while importing modules, but that was closed previously by python (once it finished reading it I suppose) and now twisted tries to do something with it and I have no idea what's wrong

Could someone please point me the right way? I'm a bit lost

Frantic: Why do you want to daemonize?

tos9: I'm trying to deamonize the run function shown here: https://gist.github.com/anonymous/514d3a7907fe4...

Frantic: Yes but *why* do you want to daemonize :)?

Frantic: The right way generally is not to daemonize.

and why do you want to do it using some third party tool instead of using twistd?

And to just use `twistd -n web`, or the newer `twist web`

tos9: I know, but my company's ops requires that something they manage detach from the terminal and daemonize

Juhaz: I don't know what you mean by "using twistd"

tos9: I mean, if it were up to me, I would have it not detach and use a process manager such as supervisord or something, but it's not up to me in this case

That sounds like quite a silly thing if they require every process to implement daemonization and not use any process managers :P

But yes, OK, if you do need to, just use twistd web

`twistd web --wsgi myapp.app` is a daemonizing WSGI container.

tos9: I have used twistd before for that, but in this case I need to do a bit of setup beforehand, for example I need to grab some info from commandline, read some config from file, etc. I suppose I could use twistd but need to figure out a way of using it from the python app

Frantic: That is what you do in myapp.app.

Or in a twistd plugin if you'd like.

tos9: But I don't know the port for example until after I run my config code

I could technically have the main python script os.system('twistd ...') but that would be hacky

So I'm trying to find a way of daemonizing just like twistd does

the config code would be in the twistd plugin

or be ran by it

Juhaz: correct me if I'm wrong, but doesn't `twistd web` take the --port option? If so, how can my module later define the port?

BTW, I'm not very experienced with twisted, so please tell me if my assumptions are wrong

I suppose what I'd like is: Given a function that configures a reactor, what's the best way to daemonize that and run the reactor

ah, yeah, I guess "web" is the plugin in that case.

I don't follow

I mean web is a twistd plugin, but since it's a ready-made one it won't run your code, I was thinking of self created plugins.

yes, I'm digging through the twisted.web code, but it's a bit hard to figure out what's going on there

tos9: I've found what you meant by doing it 'in a twisted plugin', but that means my runner needs to be managed by twistd. My app needs to run in two modes: 1) Local test run, and 2) Managed daemon run. I need to use my own runner (e.g. argparser, etc) for option 1)

OK, I've stopped using the python-daemonize package and instead wrote my own small daemonize function which does the expected things, and the twisted reactor is no longer crashing

I still have no idea what was casing the weird crash though, but whatever, I've spent so much time reading up on this that I've learned what daemonizing needs to be exactly

thanks for the ideas guys

new core enhancement https://tm.tl/#8842 by Stibbons: Readers/Writer Deferred Lock