My Jolla lasts all day when it comes to battery, sometimes 1,5 days. And I do use it a lot during the day, including a few Android apps/1 Android game. So the battery is as good as that of the Note 4 in my experience. Build quality is also very good IMHO. Old-fashioned? Yeah, maybe a bit. But not nearly enough for me to dislike the phone. The only thing that could've been better hardware-wise IMHO is the camera though I don't really care that much about it since I live by the fact that a smartphone shouldn't be used as a camera device anyway. Nothing beats my DSLR.

Even so: how is it possible to compete with the Note 4? The Note 4 is more expensive to begin with. That's like saying Hyundai needs to up their quality because Lamborghini has better quality. Totally apples to oranges. Not to mention the fact that the Note 4 is a new device while the Jolla phone is a bit older. So even if you'd compare a Note to it, then at least the Note 2 or 3 would be a better comparison (though still apples to oranges when it comes to price).
If you want to compare the Jolla phone, pick something from the same league. Jolla phone is a mid-range phone, Note 4 is high-end and Moto G is low-end.