Logistic regression updates

[krstopro]

Changes:

• Fixes loss computation by adding average over sample losses
• Fixes convergence criteria
• Adds input validation in predict and predict_proba
• Polishes the code slightly
• Adds more unit tests

Fixes #301

[josevalim]

Btw, when you have too many arguments like this, you can wrap the arguments that don't change (are fixed in the loop) in a map and pass it down. Helps clean up while loops and Elixir signatures!

[krstopro]

This works for tensors as well, right?
Apart from that, CI is failing. I am working on it.

[krstopro]

Sorry, I am not sure I understood. 😅

Would having a while loop as {result, state} = while ... where state is a map work? Is this something new in Nx?

[josevalim]

Yes, it works. It is quite old i think :)

[krstopro]

Alright, this was way more work than I thought :)
I managed to remove the optimizer completely by explicitly computing the gradients and using line search for adapting the learning rate.
@josevalim If this gets merged I would suggest releasing a new version. Logistic regression is a fundamental algorithm and there have been important changes to others (e.g. DBSCAN).

[josevalim]

It is your decision to merge it! I can ship a new version afterwards!

[krstopro]

It is your decision to merge it! I can ship a new version afterwards!

Alright then, merged. :)

[krstopro]

P.S. I am having issues compiling EXLA 0.9.2 on my M1 Mac. Perhaps we should bump the version?

[josevalim]

Released (and feel free to bump Nx as well!)