HashSet is a set, e.g. {1,2,3,4,5}
HashMap is a key -> value (key to value) map, e.g. {a -> 1, b -> 2, c -> 2, d -> 1}
Notice in my example above that in the HashMap there must not be duplicate keys, but it may have duplicate values.
In the HashSet, there must be no duplicate elements.
They are entirely different constructs. A HashMap is an implementation of Map. A Map maps keys to values. The key look up occurs using the hash.
On the other hand, a HashSet is an implementation of Set. A Set is designed to match the mathematical model of a set. A HashSet does use a HashMap to back its implementation, as you noted. However, it implements an entirely different interface.
When you are looking for what will be the best Collection for your purposes, this Tutorial is a good starting place. If you truly want to know what’s going on, there’s a book for that, too.