First, our compliments on your terminology. Efflorescence is the proper term to identify the process that caused the white streaks on the surface of your fireplace face. As the leaking water evaporated at the surface of the brick, it left mineral deposits behind in the form of a white powder.
A sponge is not heavy-duty enough for the kind of scrubbing required to remove efflorescence. Try wire brushing the area without using any acid at all. If that doesn't do the trick, then acid will be needed. The pool acid you used may have been over diluted. A ten percent solution of muratic acid (one part acid/nine parts water) is recommended. Keep the area very moist with a wet sponge while alternately scrubbing with a steel wire brush (brass will be too soft).
CAUTION: Muratic acid is dangerous. Always use eye protection, protective clothing, and gloves.
The February 1985, edition of Handyman Magazine has a terrific article on moisture, masonry and efflorescence. Write the Handyman at 1999 Shepard Road, St. Paul, MN 55116, for a copy. If you are a hard core handyman, and can afford the $60 price tag, you also may want to pick up a copy of Construction Principals, Materials and Methods , by Harold B. Olin, AIA. Building construction is covered from A to Z, and the chapter on masonry includes a section on maintenance.