If you are responding to an RFP, you will probably have to accept that everyone is taking the cost of doing estimates as part of the cost of doing business. If they respond to enough of them, and charge enough for the ones they win, this can work. If you respond to one every few years you will probably not enjoy this model. I for one tend to ignore RPFs and prefer to get work where the customer has already decided they want me to do it, but needs to know how much it will cost.

When you are starting from a completely blank slate, sell the client a “feasibility study” of two weeks or a month. (No matter how little you know about the project, you should be able to guess how long this phase needs, pretty accurately.) You will interview stakeholders, come up with an architecture and a design, perhaps do some preliminary UI design, and produce a document describing the problem and the solution, along with a phasing plan and some firm estimates (quotes even) to actually do the work. The client will own this design and architecture and is free to ask someone else to implement it. (In my experience, they never do.) You are not being “paid to estimate” – you are producing work of value and being paid for it. Everyone’s happy.

Unfortunately, getting paid for an RFP or RFQ is mostly a thing of the past. The last time I got paid for a proposal was in the late 90’s, and even then it was a dying concept. In some cases you can try to roll the proposal cost into the total project cost (assuming you win), but many customers have taken to assuming that it is their due to have you spend all of that time on their problem for free.

Your best defense is to tune your estimating skills so that you can quickly get to a ballpark figure. For example, assume you are quoting on a website. Do very quick sketches of each major page type and identify the source of the content for each component on the page. Make an estimate of the time for the frontend work, and a separate estimate for the backend necessary to generate the content. Lather. Rinse. Repeat. Oh, and don’t forget to confirm with the client about who is responsible for the actual input of the content!

If the client is more than a bit fuzzy about exactly what they need, offer to do a paid consulting session (or two) to work them through the creation of User Stories. Explain that you (and anyone else bidding on the project) will need this level of detail in order to properly estimate the scope and complexity. Sometimes this can get you in the door for a few billable hours, and it also lets them see how you think/work.

Note bene: unless you are a true God of Programming, any time you identify something you have not done before, be honest and at least double your initial estimate for that component. Or be really honest and triple or quadruple it.

I don’t think there is a single answer for your question because there are a few factors that determine how you’d approach a quote or proposal.

Requirements

Although very rare, if a client has a reasonable grasp and documentation or the project is a very simple one you can usually provide a good estimate to the client. But as everyone knows that’s a rarity and scope creep has to be factored in to your cost projects.

In this case the time required for you to prepare the estimate/proposal should be much less than if you have to determine the requirements list and scope of changes. So I’d consider this type of quote the ‘cost of doing business’. Unless your quote falls under the LOD caveat below.

You also need to clearly define what is and what is not covered in your original estimate as well as the rate and billing terms of change requests. Obviously there will be some back and forth with the client when these things come up, but a well written proposal/quote gives you something in writing to fall back on in your argument.

Level of Detail of the Proposal

How much detail are you providing the client in your proposal? If you’re spelling out anything other than the 40k foot overview you’re providing the client with a blueprint of the project requirements. In this case you’re putting a great deal of effort and man hours in to your proposal which the client can simply hand to the lowest bidder.

You’ve done a great deal of the work without any benefit or payment for your services. So charging your customer for the consultation would be perfectly reasonable.

Many clients will balk at this, but I’ve found that if I tell a client that the estimate is X dollars up front but I’ll apply that to the cost of the project if I’m selected they don’t seem to have an objection. Obviously you’d need to factor that in to your total project cost so that you’re not losing anything.

Charge for the consult. That’s all.

Of course, you will need to have developed a sufficiently good reputation that someone will be willing to pay for your advice. You can get referrals from satisfied customers, or develop a portfolio of satisfied customers to make your case.

Alternatively, notify your clients up front that, if they accept you as a vendor, the consult will be built into the quote. You should sell the fact that money spent this way is money saved in the long term by having a better solution.