#haraka

[13Haraka] 15ricardopolo opened pull request #1584: pool_timeout setting example (06master...06patch-1) 02https://github.com/haraka/Haraka/pull/1584

baudehlo_: It *might* be related to the destination mail server. The one that's having the trouble will handle ~1k messages to a single destination and maybe 1 to a different destination. So I noticed when the outbound pool hangs and a new message comes in to the 'different' destination, it gets processed and delivered.

yeah something locks up the pool at some point.

I think there's just a bug in generic-pool tbh.

And I'm surprised nobody else has talked about it.

Maybe it's this? https://github.com/coopernurse/node-pool/issues...

Probably not though since we don't set it.

Or this: https://github.com/coopernurse/node-pool/issues...

Anyway I'm seriously considering writing our own pool code.

At least it'll be easier to debug.

baudehlo_: I think it could be https://github.com/coopernurse/node-pool/issues.... I suppose one way I could test it is to lower the concurrency_max in outbound.ini to something stupid like 1 (I set it to 50 to help clear the backlog faster when I restart Haraka)....

If I had it at 50, but the Barracuda only accepted 45 simultaneous connections, it could be that ~5 are 'timing out' and the pool code is missing it.

yeah

If I had it set to 1 it might be easier to track and it might prevent the issue from occuring.

I'll see if I can get away with somewhere between 1 and 5 today and let you know what happens.

What we really need to do there is to put them immediately back into the temp fail queue.

Or only pull them out of temp fail queue if there are pool entries available.

[13Haraka] 15msimerson pushed 1 new commit to 06master: 02https://github.com/haraka/Haraka/commit/22efa...

13Haraka/06master 1422efae4 15Ricardo Polo: pool_timeout setting example (#1584)

Is outbound concurrency_max multiplied by the number of nodes running?

Ok--it is multiplied per-node.

yes

When there's mail in the queue and concurrency_max=1, they all have an 'out' section in the proctitle. After a node hangs, it no longer displays an 'out' section. One all the nodes are missing their 'out' section, outbound delivery stops everywhere.

With concurrency_max=1, I'm hitting the hang every ~30-60 seconds.

so basically where we call aquire() we should have a check for whether we're at that state yet.

I'm totally unfamiliar with the oubound code at the moment, but I'm slowly digging through it.

look at the get_client function.

[13Haraka] 15msimerson opened pull request #1587: 1575 lower case undefined (06master...061575-lower-case-undefined) 02https://github.com/haraka/Haraka/pull/1587

baudehlo, did you get a chance to look at https://github.com/haraka/Haraka/pull/1577 yet?

[13Haraka] 15msimerson opened pull request #1588: missing watch config (06master...061540-missing-watch-config) 02https://github.com/haraka/Haraka/pull/1588

[13Haraka] 15msimerson closed pull request #1588: missing watch config (06master...061540-missing-watch-config) 02https://github.com/haraka/Haraka/pull/1588

[13Haraka] 15msimerson closed pull request #1587: 1575 lower case undefined (06master...061575-lower-case-undefined) 02https://github.com/haraka/Haraka/pull/1587

@ultimatt : no because I wanted to hold off until after 2.8.9 is out.

ok. I though that might be the case.

I'm inclined to nudge you along with 2.8.9 then. ;)

and follow up with 2.8.10 when the pooling issue is sorted out

seems that might take a while....

baudehlo: Bleh. It looks like pool.acquire() is called at line 1230, but the callback (line 1314) never gets fired.

and 2.8.9 has some other important fixes

including a couple low hanging fruits I just picked

darkpixel: can you link to the lines on github as I think I have different line #s here.

The callback that never gets called: https://github.com/haraka/Haraka/blob/ea2806a60...

get_client() function that does complete successfully: https://github.com/haraka/Haraka/blob/ea2806a60...

So I think you might be right baudehlo--I think it's in the 3rd-party pool code.

That's the same lines.

Well that pool code is supposed to basically hang waiting for a pool entry to be released.

But maybe this indicates they aren't getting released properly in some circumstance.

Er...last one should be here: https://github.com/haraka/Haraka/blob/ea2806a60...

baudehlo: In my testing (I added debug code), every time I saw a delivery attempt, I saw a corresponding call to release_client.

so when it maxes, for some reason it just never gives an entry from the pool?

wtf is the point of the pool code if that's the case :)

Yeah.

I say we experiment with switching to advanced-pool, which has basically the same API.

I'm game.

...but I just got hammered with a few emergency tickets. I'm going to be AFK for a few hours.

hi. Is smtp_proxy do support LOGIN auth method? In the documentation looks so (https://haraka.github.io/manual/plugins/queue/s...), but not in the sourcecode (smtp_proxy.js)

You need to look in smtp_client.js

in the main dir.

I think this is the commit, which supposed to add support for it, but it does nothing apart of setting some configuration variable: https://github.com/haraka/Haraka/pull/872/commi...

baudehlo: hmm, took a bit of time, but looks like it is working. how to send a patch?

and: https://github.com/haraka/Haraka/wiki/Contributing

hi, I'm fed up of having no email address and I hate the discrimination email providers have against no-fos like myself. Can I ust haraka to turn my own computer into my own email server and host my own email account on my own computer?

It's very unlikely you can run it on your home computer.

why?

Most places block outbound port 25, and even if they don't, the IP will be in blocklists of consumer IP spaces.

so basically most places won't accept an email that's not hotmail gmail, facebook twitter etc

No, not exactly, but you need a server with a static IP address.

my home computer?

Well what's your home computer's IP address?

192.168.178.22

:)

go to google and type "What is my IP"

I'm sorry but I do not agree with any of google's ethics and I have decided to boycot this wicked corporation. However my public ip is 31.216.123.210

You might be OK, that IP isn't in the PBL. But /shrug

what's the pbl?

SpamHaus's list of consumer IP space.

Next step is to check if outbound port 25 is blocked on that IP. See if you can telnet to a public mail server on port 25.

well I haven't done anything bad on this computer so I don't see why it would be blacklistd

The PBL isn't about doing anything bad.

It's used to prevent botnet spam.

will I need to make an exception in my router port 25?

Not for outbound - most default router setups allow any port outbound.

For inbound, yes.

OK I'm in the router, What protocol, what port range?

TCP, UDP, ESP or GRE

tcp 25

exception adde, what's next?

oh do i wanna translate that over to port 80?

or translate that to port 25 also/

huh? Why not just see if your ISP has blocked port 25 first?

ok how do i test?

What OS?

archlinux

Try `telnet smtp.gmail.com 25`

got a non google test?

baudehlo: it is not working. The problem with smtp-proxy, is haraka responds with the smtp commands, then the client starts talking, which gets forwarded to the remote smtp server along with the authentication command.

so it is mixed haraka's smtp auth with the original sender smtp lines.

`telnet mail.emailitin.com 25`

cheers

connected :)

it said: 220 emailitin.com ESMTP Haraka 2.8.5 ready ^CConnection closed by foreign host.

it should be like this: haraka->client: smtp greetings, client -> haraka: ehlo, haraka->client: smtp available commands (also puts on hold), haraka->server: authentication, then it starts to send the clients smtp lines

currently it is mixed because the async nature of node.js

OK so has my machine passed the worthiness test? shall I go ahead and go sudo pacman -S haraka ?

HumanSheeple: yeah, but bear in mind running an SMTP server isn't trivial like running a web server. It tends to require knowledge of how email works.

arcol1: feel free to create an issue for it.

baudehlo, do you have any good guides?

instructions

haraka's not found, do u have a git link?

It's on npm. You need node.js installed for it.

pacman -S npm !?

And no, I don't know of any good guides. Most people learned from years of painful experience.

are you the creator of haraka?

well ive gone sudo npm install -g Haraka

https://github.com/haraka/Haraka this guide says i should create a service haraka -i /path/to/haraka_test what should I do with that, something like /home/HumanSheeple/Haraka_Test right?

sure

ok I'm trying to edit the host_list file

it just says HumanSheeple there

It's around about this time that you're expected to read documentation :)

got a link to the documentation for me to have a butchers?

haraka.github.io

in all honesty how long was it from the moment u installed haraka to the point u sent/received ur first email from that server?

That's not a question to ask me.

I wrote it, so from when I started typing code, probably about 4 weeks.

Actually no because I didn't have outbound support back then, so probably about 6-12 months.

how old is haraka now?

ur matt sergeant?

im watching this 5 year old video of urs called Haraka- Getting stareted Screencast, u sounded really young then

smtp_forward has the same problem. smtp_client.on in th sourceile and it is not sequencial, but rather async. so no way to makesure authentication haens irst

my smtp.ini looks nothing like the one in the video