They could find this suspicious and search harder for bogus stuff, but the point is it's your right and thus it cannot be held against you in court.

Talking too much & giving them enough materials for false testimony or pseudo-confession is way more dangerous.