Last week we had a survey.

It asked the SRS community about what they thought were the most important search skills. (Take it by clicking here, if you want to see what the original survey was all about.)

Since I do surveys professionally (for my research work at Google), I know this isn’t a perfect survey. The sample size is too small, too biased towards professional researchers, and doesn’t have a broad enough set of questions to be accurate.

I don’t care.

What I wanted from this survey is a sense of what experts think about search advice. That is, I really wanted to hear what you, gentle reader, had to say about search methods and conducting online research.

The good news (for me) is that many of the ideas you put into the survey are covered in my book. The better news is that I don’t cover everything you mentioned.

To everyone who filled out the survey, many thanks for your comments. They were thoughtful and had great insights, including a few that I hadn’t thought about before! We’ll cover some of these in future posts.

Analysis: We got 48 responses from readers. I was hoping for a bit more, but this is a great foundation to start. There were also 4 questions, so that’s 192 responses that I’m summarizing.

All I did was to read through all of the responses and summarize the themes I saw. Each of the items below was suggested by more than one person. The rank order reflects the number of times each was written about. #1 in the list was the most common, #2 was second most common, etc.

Here’s what you said, with a short explanation / summary afterwards.

This was by far the most common skill that you think good searchers should have. This makes sense, as the quality of your search query is a strong determiner of how successful your search will be. Over the years I’ve seen that some people get stuck when they can’t figure out the right search terms–that’s an important barrier for getting to a successful answer. Reformulating your query is important, both to get out of being stuck, and to hone in on what you’re actually seeking. (More on this below.)

We’ve talked about this in lots of SRS posts–a very important skill is that of reading and learning from your search experience. In classes, too often I see people not reading the results page and consequently NOT learning what worked or didn’t work. Careful reading and thinking about what your search returns leads to learning, and learning leads to better searching in the future.

Knowing what you can search for, and understanding the ways in which you can search, is absolutely fundamental. In my testing of search skills, one of the things that completely blocks people from completing their search task is not knowing that you can search for X, Y, or Z (you fill in the blanks). In one dramatic example, 250 Google engineers were unable to complete a search challenge I gave them because they didn’t know it was possible to search for archival aerial images on Google Earth. (Did you?)

When you search for something that’s more than just a simple answer, you often need to find the context that surrounds what you find. This is especially true for complex topics such as social or historical questions. Just reading a single web page (or, God forbid, reading only the snippets on the SERP) almost never gives you enough context information on the topic. Read widely, and learn the when/where/why/how about your topic.

I’ll be writing much more about this in weeks to come. My colleagues Sam Wineburg and Sarah McGrew (Stanford) have written really well about this and why you’d want to do lateral searching. See: Lateral Reading (Sept, 2017). I’ll be writing about some effective strategies for doing this.

Limiting the scope of your searching is simple, if you know how to use site: or filetype: — those are the simplest ways to get results with certain kinds of properties. You use site: when you want results from a particular web site or organization. Thus, [site:CDC.gov influenza] will give you results with a distinctly CDC perspective. Likewise, you can exclude certain sites with the minus operator. [-site:CDC.gov influenza ] Or you can search multiple sites or entire domains: [site:.IN Brexit OR site:.BE Brexit] will search for “Brexit” on Indian or Belgian sites.

Yes! This is critical… and requires much more discussion than I have room for here. We’ll return to evaluation rules-of-thumb in future blog posts.

This is a great way to find advanced, expert content on a topic. The trick here is to locate experts that are truly expert (why do you think they’re an expert?) and then tracking down their writing. The other trick here is to always locate more than one expert. Ideally, you want to see the varieties of thought on a topic (there could be more than one point-of-view… in some cases, there can be 4 or 5!), and get experts from each perspective.

A critical thinker is always wondering how could this be wrong? And how can I break this big problem down into pieces? There are many approaches to critical thinking, but these two heuristics work well for me. A critical thinker criticizes an idea, some writing, or a point-of-view… but does so productively, not to just be mean-spirited about it. Critical analysis is trying to break something into its components to understand how it works. It does not assume that authorities are always correct, but calls things into question. (This is why critical thinking annoys some people.) Of course, you want to do your critical thinking with care, and not be annoying about it. But it’s an essential skill.

I was surprised about how many people said this. Aww.. gee… thanks!

This is a variation on the “find an expert” heuristic from above. But I like it as a heuristic for asking questions too. How would an expert think about your question?

That is, ask why about the answers you’re finding. Why is this true? And How do you know? This almost always leads to a better question.

One way to create a question is to think forward to what an answer would look like. That is, if the answer looks like this, what question would let me get to that?

This is a classic library reference desk question. A great reference person always tries to dig into what you want to know.. and why. Knowing the answer to why tells you a lot about how to frame your question.

Don’t give up on the subtlties of a question. You can almost always ask “Is there more I should know?” That’s a useful question in almost all cases.