|
|
The need for "Research Quality" search engines
Opportunities for improvement -- Serving writers
This page attempts to get at the needs of writers and at the usefulness of search tools to writers. If you are a writer, you are invited to offer your opinions and comments at the bottom of this page.
Stages in preparing an essay / article / blog / thesis / research report / talk:
A lot has to happen before we begin to write.
Simple awareness of the field is a start. Next comes a recognition of scope -- What is the range of this field? What is not included? There is typically a period of immersion in the data, aided perhaps by books and mentors. For long term learning, there is a period of gestation or reflection; learning takes time. If all goes well, an internal process kicks in -- the arousal of interest, stimulated by discovering connections with knowledge already made our own.
Interest enables us to decide what we want to write about, that is, to develop a focus. At this stage, our task is to set aside material that does not contribute; less is better. As we get rid of irrelevancies, we begin to detect patterns that emerge from the data.
Creativity comes into play as we assemble a framework of insights and reflections. With a framework in place, we can allocate useful resources (quotations, references) within the framework. That done, we turn to the all too familiar stages: write, re-write, edit, polish, share the result, and attend to feedback.
Awareness
Suppose you have been given an assignment to write an article / essay on rumors of an impending reversal in the earth's magnetic field, and concerns of scientists about timing and possible consequences. Suppose the sum of your knowledge in this field is that the earth has something called a magnetic field, and that compasses point north. Period. This assignment calls for some fast learning. Where would you start? If you put either a single word or a phrase in quotes in any of the standard Internet search engines, you get a quick and reasonable response. Whoops, 832,000 hits for the phrase "earth's magnetic field". This may take some digging. Current Internet engines are good at delivering a first impression of scope in any area of knowledge that can be embodied in one word or in an exact phrase. In the year of our Lord 2008, that's one good way to start.
Scope
Each field and sub-field of knowledge has its own particular vocabulary. Would you like a vocabulary list for web pages related to the earth's magnetic field? Among the frequent words and phrases are flip, fading, magnetosphere, dynamo effect, solar wind, liquid core, iron, magnetic north, pottery, lava. Surprised? The last two turn out to be ways of measuring direction and intensity of the earth's magnetic field in times past. Lists such as this can be generated for any field as a standard output of indexing software. (An index is, after all, merely a list of words, a count of the frequency of each, and a list of positions of each word.) A sense of the scope of issues related to the earth's magnetic field can be gained, then, by either prowling around among 832,000 hits or (if you are in a hurry) looking at a few words in a vocabulary list.
Immersion, Reflection
Do you remember the teacher who asked that you read 400 pages before the next class? She aimed to immerse you in the data. This is more than simple exposure, since you are asked to reflect as you read. For learning to take place, this process requires you to be proactive, engaged, mentally involved.
Search engines can serve as aids to immersion and reflection. A major step forward would be having both Internet and personal computer searches in which the results can be retrieved in the form of new indexes. This filters out extraneous content. What remains is searchable, browsable, focused, and eminently relevant to your field of writing task.
It is useful simply to prowl around in the data, "chasing intellectual rabbits" ... looking for combinations of ideas that stimulate your thinking. Immerse yourself, reflect on what you see, enjoy.
Arousing Interest
Sorry, you are on your own here! Interest has to arise from within.
Focus, Recognition of Patterns
The very human needs to focus and to recognize patterns are critical to learning. Here we separate the sheep and the goats among search engines. A "sheep" in search engine terms delivers only precision hits ... those that are right on topic. A "goat" is a garbage delivery device ... a search engine that offers up millions of off-topic hits, most of which are too long. The good stuff is probably there, but you are left to dig for it.
To put it simply, good hits have words close together, garbage hits either have the words far apart OR they do not even have all the words you want. Good hits are meaningful. Garbage hits do nothing except get in the way as you try to focus and recognize patterns. No garbage, please. Less really is better.
Framework of Insights
You have written essays before. You know that the framework is all-important. It is essential to develop an argument, to persuade a reader, or to get good grades out of a teacher. The framework is a sequence of patterns, meaningful to you, that you see emerging from the data. The framework is inherently personal -- it is you confronting data. No two persons will arrive at precisely the same sequence of insights; each brings her or his own background to bear. The patterns that emerge, the logical sequence into which they are framed, are influenced by that unique personal background of past learning.
Allocate Resources Within the Framework
Dream along with us for a few moments. You have seen the list of tools that might make up a useful toolkit for writers and for other groups of people. Suppose all these tools were available to you, with everything except the Internet search engine on your personal computer. (The Internet version belongs on your server if you happen to operate a web site.)
For assembling resources for writing, you would likely do one or more of the following:
You can see from the above sequence that your indexes (all fully readable, searchable, etc.) become increasingly focused as you go along.
- Search the Internet using traditional search engines, perhaps increasing the number of results to be retrieved from the standard ten up to 100. (That option is found in the Advanced Search of many search engines.)
- Feed each result page as a seed to your toolkit's web crawler. Once you click start, all of the following will be done automatically: The software will interact with the web site of each the 100 results, evaluate the quality of each hit, remove content that is not relevant, and return to you the portions of pages that qualify as truly relevant. Other content (that the Internet engine rated mistakenly as highly relevant) will be ignored. The good results will be, according to your choice, either in the form of HTML tagged text or a new index.
- Search at web sites that use this same "dream" toolkit. You indicate your choice to get the results of searches at those sites in the form of new indexes.
- Download, exchange, obtain from friends and colleagues indexes of content that relate to your writing interest.
- You can at any time convert indexes into tagged text files, should you wish to cut and paste them in a word processor in order to add notes and/or isolate further the content that is helpful.
- Use another "dream" tool on your personal computer to combine these indexes into a single index (if under 3.2 million words) or a group of indexes.
- Conduct more specific searches of the result on your personal computer, one search for each part of the framework that you have created. If you take each result as an index, you have in that index all the resources for one section of your writing task.
- Convert that index into text, using one of the tools. Put the result into a word processor, and you are ready to write, with all resources for that section in place.
... And the Familiar Stages:
Write, re-write, edit, polish, share the result, and attend to feedback.
Your feedback
If you are a writer, your comments on this page would be welcomed.
|
|||||
| words close together.com | The "Research Quality" Search Engine by Marpex, Inc. | ||||