Whatever you do don't weight the flashing down - with bricks or anything else. Besides being an eyesore the "paper-weight" configuration may cause water to puddle and rust the entire flashing. You need to stop the flashing from rattling by stiffening it and, at the same time, promote water shed.

It sounds like the normal creases (stiffening bends), which usually radiate diagonally to the four corners from the hole where the chimney penetrates the cap flashing are non-existent. These creases also act to raise the center of the flashing higher than the edges, allowing for proper water shed. If the creases do exist, then we would say that chances of recourse against the contractor may be pretty slim. If you are forced to use the existing cap, you will find that the proper fix is exactly the opposite of what the roofer suggested. You need to create a constant pressure on the sheetmetal from below rather than above. The rattle will disappear regardless of what side the sheetmetal is braced from, but bracing it from below can be useful to improve watershed. Water sheds when the flashing is higher in the center than at the edges, and the chance for rust is diminished.

First carefully un-nail and remove the flashing. Then install hump-shaped wood braces at approximately 16 inch intervals from edge to edge at the top rim of your chimney. The four-foot-long braces should measure from 0 at each end and gradually thicken to about 1 to 2 inches in the center. Caution: Wood braces should be no closer that 1 inch from the metal fireplace flu. Finally, reinstall the flashing with hot-dip galvanized nails using the old nail holes. As an extra precaution, use 3M Gutter Seal at each nail head and all seams. The building code requires that all exposed sheetmetal flashings be painted (a great rust-preventive measure).