Many times, it's a ruptured diaphragm. And this can come from motor oil being sucked into the booster from the intake connection past the plastic check valve on the booster. Oil seems to rot the rubber Bendix used. Some desperate owners have TEMPORARILY fixed a torn/punctured diaphragm with silicone rubber (after alcohol cleaning), but you need to give the goo time to cure before firing up the engine. And it WILL come loose eventually- probably when you most need it- hence "temporary".
In my experience, by the time you get the booster vacuum leak fixed, the master cylinder will begin leaking fluid- if it is not already. 'New' is more dependable- either a Byars conversion or the more expensive Ferrari 308 unit.