#twisted

Lukasa reviewed <https://tm.tl/#8530>; - Delete test_webclient.py's testPartial() test, which is being skipped (assigned to rodrigc)

new core enhancement https://tm.tl/#8532 by adiroiban: Push coverage report from Travis-CI

hey guys :) jjust a little question. can i send files with self.transport.write just out of the box or do i have do decode them in some way?

insp3ori0n: That's probably not a super simple question...you can probably *generally* do it (especially if it's a binary file), otherwise you'll need to deal with some sort of text encoding.

(Although if you can control both sides that's not hard.)

clokep: i managed it now easily with file = open(...), data=file.read()... but in case of, for example pdf files this would not work

Why not?

Data is just data...

i think they cant be transfered to a string or im wrong?

with open(..., 'rb') as f: self.transport.send(f.read())

works with pdfs too?? good to know

insp3ori0n: Twisted works on "bytes" which is called "string" in Python 2, "bytes" in Python 3, essentially just a collection of data with no understanding of what it is underneath.

So if oyu send it and then save it back to a pdf on the other end, it should work OK.

thanks a lot, got it now :)

Tickets pending review: https://tm.tl/#8308, #8332, #8368, #8469 (Lukasa), #8478, #8480, #8494, #8491, #8515, #8526, #8511, #8519, #8531, #8524, #3807, #5786, #8209, #7868, #8026, #5705

so I wrote an app using twisted.web.resource

it's a web app, perhaps you would call it a web server

it has some logic that does some requests elsewhere, and computes a result based on that and returns it

some of its requests are built using NOT_DONE_YET

what are my chances of transitioning to WSGI ? (the point of this would be to deploy the app with uWSGI and be able to take advantage of the reload/restart triggered via memory utilization that uWSGI provides. because for some reason, the twisted app I wrote, leaks memory, and I am not fully confident that I'll be able to tame those leaks in due time, so I need it to be combed up and reloaded/restarted when it goes over a certain limit, for example

I am asking this in relation to a quote in the documentation, which says "Like any other WSGI container, you can’t do anything asynchronous in your WSGI applications, even though this is a Twisted WSGI container.

"

This quote somehow makes me feel that the async requests I'm doing in the twisted app (via twisted.web.client , specifically Agent) will be neutralized and forced to synchronous mode if I were to transition the twisted app to WSGI

I'm not sure if there's any truth to that, maybe someone else would like to weigh in

GitHub <noreply@github.com>; closed <https://tm.tl/#8530>; - Delete test_webclient.py's testPartial() test, which is being skipped

So, I have a program that makes HTTP requests with twisted's Agent and we're moving to HTTPS, but with a self-signed certificate. I made it work with a custom ClientContextFactory, but how would I go about making it work properly, without completely ignoring the certificate

?

adourado, i'm pretty sure treq has answered this question, maybe check it out

is always takes me a significant amount of time to get ampoule working after I haven't used it for a while :(

child processes seem to be exiting without getting a chance to do anything

the joy of trying to clean the reactor of cached connections from the http Agent.....arrrrrrrgghghhhhhhh

benbangert, if you're using the default pool there's a secret global thingy isnt there?

wait.. that's in treq

I just assume everyone uses it :P

if not, maybe you could rip it off from there

I've had to dig through so many damn secret attributes to grab this connection pool. lol

haha yeah that sounds familiar

a ridiculous amount of time spent just to shut it down, its kind of insane

glyph: fix it! ;)

FWIW this is what I'm thinking of benbangert https://github.com/twisted/treq/blob/master/tre...

ah, well, I did finally reach the connection pool, though for some reason the underlying tcp client is alive, no idea why, but twisted trials is still unhappy about reactor

ah... the whole request wrapped in a request thing

trying to find the one with the actual transport on it :/

dpn`: who/what is treq, and how do I check it out?

yea, maybe. thinking at this point of just grabbing the selectables off the reactor and nuking them

dpn`: is it on the channel logs?

it's a wrapper around Agent that makes it more like the regular python requests lib

Hmmm

I was shooting for a less radical solution

It's a "here be dragons" part of the code

=/

adourado, here's the ticket addressing that I think: https://github.com/twisted/treq/issues/28

dpn`: So I basically have to do this:

+ pemFile = filepath.FilePath('server.pem').getContent()

+ certificate = ssl.Certificate.loadPEM(pemFile)

+ customPolicy = client.BrowserLikePolicyForHTTPS(certificate)

?

I was hoping there was a way to do it without needing to distribute the certificate with the application

adourado, you could fetch it from somewhere on app startup

you have to distribute it somehow...

Got it

Seems like we'll actually have a proper certificate for the production environment, the self-signed one is just for test

I suppose it would work fine them, with a proper certificate? The docs on the Agent class says it supports HTTPS transparently

adourado, generally it should be on par with whatever browsers use

so, comparable to curl i suppose? it's been a while since I looked at the specifics

dpn`: got it. Fingers crossed =)

FWIW we use a proper cert for our staging env too

Tickets pending review: https://tm.tl/#8308, #8332, #8368, #8469 (Lukasa), #8478, #8480, #8494, #8491, #8515, #8526, #8511, #8519, #8531, #8524, #3807, #5786, #8209, #7868, #8026, #5705

either as a subdomain or similar domain to our prod env

dpn`: checking it now with team leader. I'm betting we won't use proper cert on dev

new core enhancement https://tm.tl/#8533 by rodrigc: Mark twisted.internet.defer.TimeoutError as Deprecated

yea, I can't kill the damn thing

gonna have to reach into the reactor and remove the selectables via extra gross mechanisms

benbangert: I guess you've tried agent.pool.closeCachedConnections() and waited on the deferred it returns?

yup, no help

did stopListening on the server, chained its deferred to closeCachedConnections.... dirty reactor still present

maybe there's more pools hiding out somewhere

this is the raven client's twisted transport, so its kinda goofy

benbangert: Can you see what kind of object is being waited on?

Is it perhaps the server protocol?

https://jml.io/pages/how-to-disconnect-in-twist... is perhaps relevant.

benbangert, are they connections that are being held open by naughty web servers?

there's an odd traceback

(are you hitting third party servers?)

nope, its a localhost test