Edit (solved for real this time I think):

I finally got numbers working, but I got the fix wrong in my first post. Of all the things I tried, it turned out to be this modified setxkbmap command that was the real solution:

setxkbmap -device 4 -I -I/usr/share/X11/xkb-chinook -rules base -model evdev -layout us -print|xkbcomp -i 4 - $DISPLAY

Confirmed that you can plug in pc105 instead of evdev and the same line also works, so it isnt an evdev or pc105 thing. I also tried all variations without piping through xkbcomp. It appears that piping through xkbcomp is what fixes it. In my case, I dont even need to hit a key on the internal keyboard, it just starts working right away with numbers.

I executed that command as a user, with the internal keyboard open.

For anyone curious, the solution that I 'thought' was responsible for the fix, was to type the normal setxkbmap command on the internal keyboard and then quickly close the internal keyboard right after hitting enter. When I tried that solution again today, it did not work, so I retraced my steps and it turns out that the diagnostic line above was what did it.

I have a qt app installed that someone made that auto launches the setxkbmap command, so I am in the process of getting rid of that to make sure that the combination of that script and this command are not jointly the solution. It also appears that a new bt-hid-scripts package has been released, so who knows, maybe this fix is in there? That sure would be fast