It is almost as (un)safe if one just ssh in as root.
You have probably gainroot installed, so getting user access means root access also.
Actually having two passwords which either one leads to root access is less secure than having only one.