I ran a full race, manual brake set up on the street in my '68 GT350.
I was using a 1-1/8" bore master.
The brake pedal was firm but didn't require much travel and there was no issue with leg effort driving it on the street.
The stopping power was really effortless and it seemed like I could not lock up the brakes at all.
The car stopped and it stopped straight, so I would comment that it is possible to use a fully manual brake system with no power assist safely and effortlessly on the street.
It is just a question of having the correct components together.
Those were with Porterfield r4s street pads and there was less pad wear with them.
The only issue, if it was in fact an issue, was those pads do generate more dust on the wheels.
You are re-engineering the system so expect some quirks that you must adjust for.
In the Pantera I would expect any kind of larger rear brake pad/caliper combination to need the pressure to them dialed way down. Where it winds up to your satisfaction is up to you.