This is a naive but typical question. The answer is that both forms are used. If you start a publication, you will probably establish something called a "style guide." In it, you will make CHOICES. You will decide what styles you support. When an issue comes up, you will describe the issue and how you decided to resolve it in the style guide. The reason you have a style guide is that inconsistent decisions on such matters can be jarring and distracting to readers.

I think "styluses" sounds less artificial and nerdy than "styli," so that's what I use and support.

There is no final answer, but I think that "styli" is going the way of the dodo bird.