Research in the Era of Bad Search Results - In search of the new "Google Fu"

[melty]

My thinking lately is that the new "Google fu" is knowing ahead of time which specific sites you need to go to in order to actually start your search. Which is completely backwards, but that's where we are.

For me personally it's not tech questions, but following online trails. I cannot get decent search results for people's usernames, but there are "username availability checkers" where you can pop in usernames and they will pull up all the sites where those names are taken. You can then go from there and check those sites individually. That's just one example of needing to know beforehand where to start your search.

You also need to know how to search all the major social media sites individually. This has been a problem for a little while as you have always needed to go to Facebook to search Facebook, but now it seems to be an issue with other and outdated networks where it wasn't before. Google doesn't seem to want to show a lot of personal information or content generated from individuals, even if you're specifically looking for that. I add "reddit" to a lot of queries because, sadly, reddit has much better answers to my questions than the woke corporate news sites that Google always points me to first. But I'm starting to worry that is going to go away as well. Reddit's search isn't great and it's unlikely they want people to stumble upon any unsavory subreddits they haven't gotten around to banning yet.

I think people like us are going to be forced to go back to the 90s/early 00s solution of basically compiling our own links pages that link to sites that are actually useful and sharing them, unless duckduckgo improves further. I also use the duckduckgo first, Google second method now and duckduckgo is no longer noticeably worse than Google.

 

[protomartyr]

why does google never appear in Searx results? I've used it many times and all of the results are from DDG (literally all of them).

Late but this sounds like a problem on your end. I’ve used Searx quite a bit recently and I always get google & bing results in the first few entries.

 

[Kendall Motor Oil]

Searx is basically a condom for web search, allowing you to search the big engines like Google, Bing, etc and get back direct links rather than the tracking links they serve up. You can even self host it, so throw it on a VPS or a home server connected to your VPN and you've effectively got anonymized search.

I'd imagine something like this could be modified to filter out known spam domains, sites that invade your privacy with shit like tracking pixels, inconvenience-as-a-service type aggregators like Pinterest, etc. This could even possibly be done using a subscription list model a la AdBlock - precompiled community maintained lists you can add/remove/modify entries from for a tailored search experience.

I also came across YaCy recently, but I haven't vetted it enough to endorse it yet. A peer to peer search engine seems like a great place to start, if participation can be drummed up enough. I would say I'm pro-federation for something like this, but we've seen how that goes.

I self host a searx in a datacenter and its good enough for me to solely use at home. The image and video search returns satisfactory results reminiscent of older search engines. The themes are rather lacking and you can run into an issue where google and bing will flag the instance, triggering a captcha request. Searx doesn't have a pass through feature yet so you may have issues self hosting. Invidious has the same issue but they have a feature that lets you pawn the captcha to some pajeets for a couple cents.

Yacy is all over the place in terms of quality. You will get porn results from small porn and spam sites. It's worth trying a public instance out to get a feel. It may be worth using as a last resort. The documentation is extremely bare bones for both search and hosting. You cannot use it to catalog local files unless all of those files are smaller than the amount of ram the program is allocated since yacy tries to read the file into memory. It is a neat concept but it falls flat in the standard euro open source way.

Google changed their time-restricted search a while ago but I still find it useful to eliminate new results in its gimped state. I guess Google gives priority to newer/updated pages.

why does google never appear in Searx results? I've used it many times and all of the results are from DDG (literally all of them).

Google may not be configured as a default engine for the instance you are using. Check in the preferences.

 

[bippu_as_fuck_ls400]

I cannot get decent search results for people's usernames, but there are "username availability checkers" where you can pop in usernames and they will pull up all the sites where those names are taken. You can then go from there and check those sites individually.

sherlock (supported sites) has worked well for me in the past for username checking. I haven't used it in a while though.

 

[Bongocat]

Not really a solution to the problems listed here, but since the !google tip, I've found searx useful for using google with a vpn. Google hates the shit out of VPNs and incessantly recaptchas you, so the 'search condom' method is useful here.

 

[Jarolleon]

The question thing is because most people put in questions. Back when this was less effective, getting someone to just use keywords was pretty much impossible. 100% impossible if the person used speech to text.

So has search gotten worse because it's being optimized for speech to text questions and we're used to it being optimised for desktop computer users?

 

[Bins]

So has search gotten worse because it's being optimized for speech to text questions and we're used to it being optimised for desktop computer users?

Back in the late 90's/early 2000's it was speculated that due the availability of internet, and computers in general, later generations would be smarter and more tech savvy.

Turns out search is probably being optimized for a generation that puts Gorilla glue in their hair for views.

 

[AmpleApricots]

The problem with such predictions always is that the average person is an idiot and expecting them to be proficient at anything is often wishful thinking.

 

[Temporal Refugee]

Back in the late 90's/early 2000's it was speculated that due the availability of internet, and computers in general, later generations would be smarter and more tech savvy.

Turns out search is probably being optimized for a generation that puts Gorilla glue in their hair for views.

That's because the speculation was based off of the current (at the time) reality, where the internet was largely a way to access information for people who would seek out information. Nobody anticipated the ubiquity of technologies like high speed wifi and cellular data or smartphones, nor did they predict the similar ubiquity of banality brought on by services like Facebook feeding peoples egos and short circuiting our reward pathways.

We thought curiosity and a desire to access information were our passes to get through the toll booth and onto the Information Highway, but didn't realize Google, Facebook, et all would effectively take a wrecking ball to Yahoo/AOL/Ask and MySpace to pave over them and build an on ramp with free* admission. Suddenly, idiots and geniuses had not only equal access to these resources (predicted), but (unpredicted) equal influence in shaping its future because access and community standards were suddenly under the control of entities operating under a business model that indulged the worst in us, at a price that couldn't be competed with by the old guard (free), with a complete focus on quantity of users (to farm data from) rather than selecting for profitable customers (paid services) or contribution to a community (forums, chat rooms, and the like).

We expected Eternal September to finally end as the last freshmen joined us and then slowly moved towards seniority status, but then we got a new former frat bro Dean who basically decided to turn the entire school into one big kegger, and encouraged everyone to become "career students" with no ambition of growth or graduation. The resources for tool development went towards catering to impulse and ignorance, rather than enlightenment, because a few very clever people figured out how to make a shitload more money off of how most people are, rather than an ample but far lower amount off of how some people are, or all people could be.

So here we are, in their world instead of the other way around because bread and circuses both cost less to produce and sell far more tickets than a symposium. We somehow thought the people offering us near miraculous convenience at "no charge" were no different than those we'd been doing business with for years who offered clearly defined services with plainly stated price tags. By the time we started waking up and realizing the deal was too good to be true, we were no longer a statistically significant demographic in the new market, or the new culture.

Another way to look at it is that those of us bemoaning the passing of the internet as we knew it were simply fortunate enough to reap the fruits of an internet cultivated by academia, whether we took part in its formation or not. Little did we realize that the same trend in increasing accessibility and commercial influence that facilitated the growth of internet culture into what we loved would continue to import more and more of the "real world" until it inevitably reflected the worst of us alongside the best of us.

We experienced the childhood and teen years of the internet, optimistic and full of potential yet painfully unaware of the changes and events that would inevitably come along and strip away possibility in favor of certainty. The internet went from a bright eyed kid who wanted to be an astronautical engineer or a physicist when it grew up, to a moody nerdy teenager who'd stay up all night on Fridays discussing science between rounds of CoD or beer pong, to a jaded doomer chain smoking cigarettes under the stars behind the 24 hour call center on its lunch break, because aspirations don't pay the fucking bills.

 

[BootlegPopeye]

'Why the internet became what it did' could (and maybe should) be the subject of a whole other thread, but I think @Temporal Refugee is right. What I'd add with the birth of social media, starting with myspace and then facebook, people started making fewer and fewer originally websites that were non commercial - anyone remember when there used to be more fan sites that weren't on gay Wikia software?. With Reddit, people stopped creating as many forums. With the simplicity of a ready made profile/forum/whatever, you had the internet effectively reduced down to a couple of dozen websites for most users, which is in part why we have the tech-opoly we do now.

More than anything though, I blame smartphones for normie-izing the internet. It's hard to imagine it now but there was a time when you never saw memes IRL. Now the glorified bloggers who style themselves 'journalists' at places like the Atlantic will have 800 word essays on the latest meme.

 

[MarvinTheParanoidAndroid]

Yep, I've definitely noticed this. I was just trying to find out the name of the college professor who claimed his girlfriend was a psychic/genius (I don't remember which but it barely matters at this point) and he wouldn't let his colleagues test her to prove his claims, and all Google gives me are articles about college professors confessing to white privilege. Then I tried using Bing instead, and somehow got several articles about Donald Trump. I tried DuckDuckGo and no dice, although at least the results don't serve me "DONALD DRUMPF." but it still gives me garbage results like "Sheldon Cooper" and "What Judges Need to Know About Narcissistic Personality Disorder"

Am I seriously about to use Yahoo?

 

[melty]

Yep, I've definitely noticed this. I was just trying to find out the name of the college professor who claimed his girlfriend was a psychic/genius (I don't remember which but it barely matters at this point) and he wouldn't let his colleagues test her to prove his claims, and all Google gives me are articles about college professors confessing to white privilege. Then I tried using Bing instead, and somehow got several articles about Donald Trump. I tried DuckDuckGo and no dice, although at least the results don't serve me "DONALD DRUMPF." but it still gives me garbage results like "Sheldon Cooper" and "What Judges Need to Know About Narcissistic Personality Disorder"

Am I seriously about to use Yahoo?

I just tried looking for this in Google and also got a bunch of results about white privilege professors. What the fuck?

 

[MarvinTheParanoidAndroid]

I just tried looking for this in Google and also got a bunch of results about white privilege professors. What the fuck?

Yeah, the particular case I'm looking for is of relative fame too, not some obscure shit on LiveJournal from the early aughts of the internet. Does anybody happen to know the name of the professor in question?

 

[Kosher Dill]

I was just trying to find out the name of the college professor who claimed his girlfriend was a psychic/genius (I don't remember which but it barely matters at this point) and he wouldn't let his colleagues test her to prove his claims

Kind of reminds me of the story of L. Ron Hubbard's first "Clear":

Finally, L. Ron Hubbard’s first ‘Clear’ — Sonya Bianchi — found again after 68 years | The Underground Bunker

tonyortega.org

But even if it's not, it's a story worth your time.

 

[MarvinTheParanoidAndroid]

Kind of reminds me of the story of L. Ron Hubbard's first "Clear":

Finally, L. Ron Hubbard’s first ‘Clear’ — Sonya Bianchi — found again after 68 years | The Underground Bunker

tonyortega.org

But even if it's not, it's a story worth your time.

Not the same guy, I remember specifically it was a college professor and his student girlfriend.

Oh and I tried Yahoo, it's just as useless as Bing, Google and DuckDuckGo.

 

[Pinochet Was Right]

I tried searching for the professor thing. I only got shit about movies and novels with no clear pattern.

I've never found shit in DDG at all. Every single result is completely non-sequitur.

Just a few hours ago I wanted to search for low budget low spec pc builds, not to play games but for home casual use, however since you know, 99% of the PC builder sphere revolves about 1337 rGb eDgY gAeMrZ, I also wrote in "nongaming". Guess what google gave me: Pages upon pages of "games that can run on low spec nongaming pcs".

So I ran the same search on Bing and voilà, ACTUAL TOPICAL RESULTS.

 

[Annealing Crater]

The last few years I’ve really started to notice google altering my searches. Using technical terms and having the search autocorrected into a similarly spelled but completely different word. I could swear it sometimes gives the same results even if I click the “no I really did mean to search for that” link. Or having random search terms excluded from the results. Sometimes it even seems to add in terms to the search that weren’t included, like it thinks they’re more appropriate, but they never are. You can see it sometimes with the bolded words in the results.

 

[Pinochet Was Right]

Two more things Google always does: Songs! I always search for phrases just to see what do people think about a certain issue. God forbid my query even RESEMBLES the title of a song, popular or not! All my results will be music shit about That Song Whose Title Resembles What I Typed. Hell, the other day I was researching a certain industrialist from my country... and the top result was an anticapitalist song named after the guy, from a literally whomst anarchopunk who put his noises in Spotify.

Finally: My native language is Spanish. Lately, when I search for stuff in that language I get results from Spain, especially when looking to buy stuff online. I live in Latin America. It's like Google forgets there's a huge location flag set up with my country. This is like going full circle to 20 years ago when all online services had practically no presence in my continent and it was understandable you'd get a lot of results from Spain because internet was new here.

On a tangential note, a couple of days ago I was looking for some Facebook tech support, and I searched in English because people who speak my language are retards when it comes to tech and asking tech questions. I clicked a link to FB's own tech support site where someone was asking the same shit I was looking for, and I get redirected to a patronizing page in Spanish saying "it looks like you're visiting us from [shithole country], we're taking you to your language's tech support page, have a nice day"

 

[longtimelurkerfirsttimepo]

Two more things Google always does: Songs! I always search for phrases just to see what do people think about a certain issue. God forbid my query even RESEMBLES the title of a song, popular or not! All my results will be music shit about That Song Whose Title Resembles What I Typed. Hell, the other day I was researching a certain industrialist from my country... and the top result was an anticapitalist song named after the guy, from a literally whomst anarchopunk who put his noises in Spotify.

Finally: My native language is Spanish. Lately, when I search for stuff in that language I get results from Spain, especially when looking to buy stuff online. I live in Latin America. It's like Google forgets there's a huge location flag set up with my country. This is like going full circle to 20 years ago when all online services had practically no presence in my continent and it was understandable you'd get a lot of results from Spain because internet was new here.

On a tangential note, a couple of days ago I was looking for some Facebook tech support, and I searched in English because people who speak my language are retards when it comes to tech and asking tech questions. I clicked a link to FB's own tech support site where someone was asking the same shit I was looking for, and I get redirected to a patronizing page in Spanish saying "it looks like you're visiting us from [shithole country], we're taking you to your language's tech support page, have a nice day"

"Please stay in your containment zone"
-Facebook

I'm sorry that all I know about the entirety of Latin America is tacos and sombreros

 

[BootlegPopeye]

This is not quite the same as regular Google search, but has anyone else noticed how much Google has completely fucked Google Groups ? The search is a complete joke now, even with the 'advanced' search you can't find shit, can't sort anything, leaving you to scroll through potentially hundreds of pages of results.

I need a better way to search old Usenet posts.