How can I determine B_max from a homologous competitive binding experiment (where the hot and cold ligands are chemically identical)?

"Homologous competitive binding " is a term we use to refer to competitive binding curves that use a single concentration of radiolabeled (hot) ligand, and various concentrations of the identical unlabeled (cold) ligand. The analysis assumes that the system is at equilibrium and that both hot and cold ligands bind with identical affinities.

Chapter 38 of Fitting Models to Biological Data using Linear and Nonlinear Regression explains how to fit homologous binding data. This 350 page book comes with Prism 4, and you can view and print the pdf file (free).

The best-fit value of the B_max will almost always be a lot higher than the binding you observe with no competitor. This makes sense. The B_max is the maximum binding at very high concentrations of labeled ligand. In your homologous binding experiment you probably used a concentration near the K_d, so it binds to only a fraction of the sites.