A reference source is any source you use for a quick look up of information: a dictionary, a directory, an atlas, an encyclopedia, etc. You would not normally read a dictionary from cover to cover or spend a weekend flipping through a telephone directory: you use these sources to find a specific answer to a question.
For research purposes, using a reference source as you start out is a very good idea. As I discussed in the Subject vs. Topic and Keywords tabs, when you are first researching a topic you should find out key names, key organizations, key dates, etc. so you can incorporate those terms when searching for more detailed books and journal articles.
You can take two routes to finding reference sources at the library. One is to do a OneSearch search and pick "Reference Resources" under the "Resource Type" heading on the right hand sidebar (you usually have to click on "Show More" to pull up the reference resources). The other route is to go to the library page, click on Databases, click on Type and pick "Dictionary, Thesauri, Encyclopedias."