While in modern English, the term "bugbear" is really only used to mean a source of irritation or a recurring problem, it did originally refer to a kind of supernatural being. The first half of the word likely has the same origin as the word "bogeyman", and implies a sort of evil spirit. Whether or not the second half was originally meant to indicate that the spirit in question looked bear-like is less apparent, but it was certainly used as such in the Late Middle Ages. By that time, bugbears, like the bogeyman, seem to have been used more as something to frighten children into obedience (such as, say, not wandering off into the forest) than as anything seriously believed in by adults.
The D&D conception of the bugbear is initially an exceptionally large and hairy goblin. Its function in the original editions is as a means of extending the list of five "evil tribal humanoid" races one further step, and thus providing a continuing challenge once characters had reached 3rd level or so. As with those other races, they have become more detailed and varied in use since.
1E
In 1st edition, the five tribal races of evil humanoid form a distinct game mechanical hierarchy, running from kobolds to gnolls. Bugbears are essentially the same idea continued one step beyond the gnolls. In fact, almost the only difference between the two races in terms of game statistics is that the bugbear has one extra hit die, and thus a 5% higher chance of landing a blow in combat and (on average) 5 extra hit points.