The Art of Searching for Open Source Intelligence

The Internet is a big ocean, and it carries loads of information you might be interested in or looking for, but where and how to find that information? Thanks to search engines like Google that make the searches using a query possible, but is it enough? If you think so, then you need to rethink about it, we have world wide web, deep web, and dark web, are you getting information from every corner of the Internet? Well, it depends on how you are searching particular information; this is why it calls “art of searching.”

The art of searching applies to many aspects, but here it is being used particularly for open source intelligence. The objective of this series is to discuss the open source intelligence (OSINT) concept, tools, methodology and processes, the art of searching is the part of this series, in this series, we will look at the Internet using different prospect, and we will utilize a different approach.

Internet Research Experts is a Myth:

People claim that they are Internet research experts, and they can find anything using their techniques, but nobody knows about the actual information available out there. Internet changes constantly, it changes within a fraction of seconds, while you are reading this there have been significant changes occurred on the Internet. The second reason can be drawn from the size of the Internet; it is huge, and you can’t claim that you have scanned every corner and provided the correct information with the accuracy.

You always find the information you are looking for on the Internet, but it changes so the accuracy can’t be measured. It makes open source intelligence a continuous process; World Wide Web is not the Internet. However, it is part of the Internet. The Internet is the network of different networks; it is an umbrella of the connected devices (computers, printers, routers, switches, servers, etc.) Think of the shodan; it does not search web pages, whatever the devices. Think about the quick port scan against any technology infrastructure; you scan for the devices, not the web pages.

So the important points to conclude here:

  • Open source intelligence is a continuous process and organizations should adopt it like this.
  • The World Wide Web is not the entire Internet; you should search the Internet for the information
  • War dialing is not dead, it evolved and changed its shape (think of the random port scanning)

New School Open Source Intelligence

Intelligence gathering is not a new topic, people of every era used to gather intelligence using their own techniques, now the techniques have been evolved, and we have ‘new’ way of searching the information.

The word ‘open’ refers the publicly available sources; it has nothing to do with the open-source software. “Open source intelligence (OSINT) is the process of collecting the intelligence from publicly available sources, paid or free, print or electronic.” The scope of OSINT is not limited to cyber security only, but business and corporate intelligence, military intelligence and another field where information matters.

Businesses hire information brokers and private investigators to gather information of their competitor, well it connected with the competitive intelligence; a corporate term refers the process to gather information about the competitors. The process utilizes the same techniques of gathering information:

  • Web-based communities: social media websites, forums/blogs, Wikis, video and images sharing websites, news portals and other user generated mediums
  • Dark web
  • Newspaper, magazines, radio, television and computer-based information
  • Govt reports, press conferences, marketing surveys, speeches, press releases, official statements (Tweets, Facebook posts, etc.)
  • Academic research papers, theses, and dissertations, and interviews

Military and security agencies use open source intelligence to counter the terrorism and to gather the information of their opponents, content analysis of Middle East’s regional newspapers are mostly effective to predict the stability/instability of the region. It is an effective technique of gathering the cultural and demographic intelligence from the area not covered by the military intelligence spy. Commercial imagery sources and digital maps provide the up to date information to military commander regarding the airfield, roads, bridges, buildings and Govt. offices.

Offensive VS Defensive OSINT

As discussed OSINT has a broader scope, but this series primarily focuses on cyber attacks. Offensive OSINT is when you study the attack before it happens while defensive is learning about the attacks against a company. The OSINT gives opportunities to both the defender and attacker; you can learn the weakness of a company and fix it while at the same time the weakness could be exploited.

The OSINT Process

In the first step, you need to identify the sources from where you can get the required data, there are many techniques of acquiring the data, but the identification process is the most important because this is where you decide the result of the overall activity. Every single step of the process will be discussed throughout this series.

Harvesting divided into two types:

  • Active harvesting – Target can learn about the harvesting
  • Passive harvesting – It makes no connection with the target, hence targets never know about it

What information to look

  • Technology infrastructure

    • IP
    • Hostname
    • Services
    • Networks
    • Software / hardware versions and OS information
    • Geo-location
    • Network diagram
  • Database
  • Documents, papers, presentations, spreadsheets and configuration files
  • Metadata
  • Email and employee search (name and other personal information)

The information above can lead to the following cyber attacks:

  • Phishing
  • Brute force (password)
  • Denial of service
  • Social engineering

The search engine seems a rich source to search particular information; however, it is not in many cases. You can’t find the classified information by just Googling, yes, using Robots.txt you can make stop the crawler to crawl and index a particular page. The search engine can only provide the pages that it indexes. Apart from search engines, utilize online libraries and private forums/blogs.

Students study Boolean logic while studying digital electronics or other related courses; this logic also applies to a search engine, operators, and the relationship are: AND, OR and NOT.

Quotation mark ” “

It shows the exact order and phrase

For example: “African Americans.”

-word to exclude it “African Americans” diet -kid -girl -“marriages.”
AND, a default operator If you write ‘infosec training’ or ‘infosec and training’ it makes no difference. Use AND with other operators. Otherwise, it makes no difference; it shows the result where every type keyword occur
OR allow more than one term It does not follow a specific order. However, it requires at least one of the term to appear in the result

“African Americans” OR blacks

Meta Search Engines

Yippy Clustering Search Engine

Yippy is formerly known as a cluster, and it is the best amongst the meta search tool available so far. It is unique because it employs its own clustering engine, software that organizes unstructured information into hierarchical folders. Clusty offers clustered results of Web, news, and certain specialty searches. The Clusty default is to search the web using Live Search, Gigablast, Ask, Wikipedia, and the Open Directory. Let see the logical categories that it creates:

It creates the cluster without the correct spelling or not

It allows the user to look at the sources of the search results and types of sites (e.g., .com, .qov). It also supports all the advanced search queries, and operators discussed before. It automatically groups huge amount of information logically at the same time it also shares the new areas of subject development. It also allows the user to create a custom tab based on the need of the search result; you can select the news sources, directories, and particular domain extensions.

Keep in mind that no search engine is the best and use more than one search engine during your research, utilize the specialized search engine for specific cases.

Other Metasearch engines:

Geographically limited search engines:

Naver Korea
Biglobe Japanese
Search.ch Switzerland
Walla Israel
Najdi.si Slovenia
Maktoob Arab
Rambler Russia
Yandex Russia, Turkey, Ukraine, Belarus, Kazakhstan
Ansearch Australia

Search documents and files

There are numerous people search engines are also there, but we will be discussing them in detail while doing the analysis and we will see how an attacker get the valuable information to launch their phishing and social engineering attacks.

Accessing the darknet for the information is crucial now a day, people share valuable information behind the TOR project, but during the open source intelligence process, you need to dig around everything and anything. You can access the darknet using the TOR, but you must require a smart darknet search engine like ahmia.fi to look for the relevant information in the darknet.

Ethical Hacking Training – Resources (InfoSec)

Advance search operator – Google

Operator Description Web Images Groups News
Intitle Search page title yes yes yes Yes
Allintitle Search page title Yes Yes Yes Yes
Inurl Search URL yes yes no Not really
Allinurl Search URL Yes yes Yes Not really
Site Search specific site Yes Yes No Not really
Allintext Search text of page only Yes Yes Yes Yes
Filetype Search file Yes Yes Not Not really
insubject Group subject search Like intitle Like intitle Yes Like intitle

Search engine is so powerful, and sometimes it shows the information that should not be available for public, a basic search operator:

Inurl: admin URL: orders

It is too dangerous for a company, and they should think about it.

Intranet or private network of companies is open, although they should be protected, this is what we call vulnerability? A quick Google search “Welcome to Intranet” reveals many addresses of the intranet that could be exploited. There are hundreds of examples where hackers used a search engine to find the important information about a company. Another example shows how someone can get the username and password of NOD32 antivirus by just Googling “intext:”eav” filetype:txt”

This is certainly not the end of the art of searching, not even the end of operators. Google hacking database provides lists of operators (dorks) to use for different purposes. In this article, we have discussed the OSINT from the search point of view, but there are so many things to discuss including but not limited to metadata searching, people searching, technology infrastructure and how it linked with an attack. Well, the next article of this series will focus on these objectives.

Source

Sourcegraph: An Open-Source Source Code Search Engine

Last year, the Code Search and Navigation tool ‘Sourcegraph‘ was declared Open Source. As it makes navigating through Source Code much more convenient, the tool itself going Open Source was definitely a big plus for developers!

We’ve looked into its features and also tried to find out how it can be so helpful for developers who are used to navigate through code hosts like GitHub, GitLab and others quite regularly.

Now, with its new 3.10 release, let us take a look at how it helps and what’s new.

Sourcegraph Features

As stated on their GitHub page, Sourcegraph has the following features:

  • Fast global Code search
  • Intelligent Code recognition
  • Code host Enhancement on GitHub, GitLab and more
  • Extension API for easier third-party integration

New Features in Sourcegraph 3.10

You will find the latest Sourcegraph 3.10 added with significant improvements and features.

The key highlights for the latest release as mentioned in their official blog post are:

Improved search autocompletion, native GitLab integration, and search and replace automation campaigns

Let’s take a quick look at what has changed with Sourcegraph 3.10:

  • Sourcegraph now provides native code intelligence to GitLab
  • Improved autocompletion for search query filters
  • The ability to create cross-repository search and replace campaigns
  • LSIF-based precise code intelligence now supports five languages: Go, Typescript, Java, C++, and Python
  • Fully automated release testing process

For more information, you can view the complete changelog.

You can deploy Sourcegraph on your server and configure it to work with your or your organization’s Git repositories. Once that’s done, you get a search engine where you can search all the codes.

But if you are a lone developer, like me, you can still use Sourcegraph on GitHub or GitHub alternatives like GitLab

I am going to quickly show you how to use Sourcegraph for better code navigation on GitHub.

Using Sourcegraph on GitHub

Let’s find out how you can easily try this tool with a Firefox or Chrome extension. Here is the official extensions page.

Sourcegraph Official ExtensionsSourcegraph Official ExtensionsSourcegraph Official Extensions

Security Issue Open on GitHub

During the first half of June, I reported a security issue to Sourcegraph that is now currently open on GitHub. Apparently, the issue was fixed around a week ago but yet to be merged to the master branch. The issue is about the official Firefox extension that asks for access to all websites during installation while there should be a filter for repository sites only such as GitHub, GitLab, BitBucket, and others. Originally, due to a Firefox bug (now fixed), it could not be incorporated during early development.

This is how it looks like with the official Sourcegraph extension installed and when you view a file on the Vim repository on GitHub:

Sourcegraph source code engineSourcegraph source code engineSourcegraph extension on GitHubSourcegraph extension on GitHub

Note how we can see the new Sourcegraph buttons within the GitHub interface, thanks to the installed extension. One thing to note is that one need not even login into GitHub to navigate through hosted Code and their repositories in order to make use of the helpful features of Sourcegraph.

When you click on “View File”, the entire look changes and the file is opened for you in a completely new interface within the browser itself:

Sourcegraph extension on GitHubSourcegraph extension on GitHub

Without Sourcegraph, if you want to look for files with a particular format, say C++ .cpp files in this example, it is very difficult to filter and view them if we try to use GitHub’s own search engine within this repository:

Sourcegraph extension on GitHubSourcegraph extension on GitHub

But once you are using this extension, see how easily you can view all such files in one go within the repository:

Sourcegraph extension on GitHubSourcegraph extension on GitHubSourcegraph extension on GitHubSourcegraph extension on GitHub

Sourcegraph can narrow down through Code Search very intelligently as explained in this video:

Code intelligence in Sourcegraph is powered by Lang Server, which enables identifying the type of Programming Language you are using:

Language support SourcegraphLanguage support Sourcegraph

Learn more about its usefulness in the following video:

Bonus Tip on using Sourcegraph 

Even without installing an extension on your browser, you can directly use Sourcegraph as an IDE on top of any repository on GitHub by just adding “sourcegraph.com/” as a prefix to the repository URL.

For example, the URL for the official Vim repository is:

github.com/vim/vim

To view the same through Sourcegraph, modify the URL as below and you’re good to go:

sourcegraph.com/github.com/vim/vim

I’ve also tested this method with GitLab and it works there too! You can try other repositories as well! There is an unofficial extension called Open on Sourcegraph that uses this method on Firefox and Chrome.

Sourcegraph Developers have a master plan behind declaring it Open Source:

Make basic code intelligence ubiquitous (for every language, and in every editor, code host, etc.)
Make code review continuous and intelligent
Increase the amount and quality of open-source code

Here are the ways they suggest you can contribute to its Development:

So this was a brief look into how Sourcegraph can make the developer’s life a lot more easier and hassle-free. You may also want to take a look at Sourcetrail, a recently open source project that allows you to visualize code base.

Are you a Developer? Would you like to adopt this new Open Source tool in your day-to-day programming tasks? Let us know in the comments section below.

95 Shares

Source

Open Source Search engine – AlternativeTo.net

Trying to find anything on the Internet without a search engine would make finding a needle in a haystack seem like child’s play. Without ubiquitous indices such as Google and DuckDuckGo, our access to the vast sea of information and content that the Internet hosts would be severely limited. This pages lists all of the major search engines that you need to know in order to access most of what the Internet has to offer.

Source

Open Source Search Engines in Java

Egothor

Egothor is an Open Source, high-performance, full-featured text search engine written entirely in Java. It is technology suitable for nearly any application that requires full-text search, especially cross-platform. It can be configured as a standalone engine, metasearcher, peer-to-peer HUB, and, moreover, it can be used as a library for an application that needs full-text search.

Go To Egothor

Nutch

Nutch is a nascent effort to implement an open-source web search engine. Nutch provides a transparent alternative to commercial web search engines.

Go To Nutch

Lucene

Jakarta Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text search, especially cross-platform.

Go To Lucene

Oxyus

Oxyus Search Engine is a Java based Application for indexing web documents for searching from an intranet or the Internet similar to other propietary search engines of the industry. Oxyus has a web module to present search results to the clients throught web browsers using Java Server that access a JDBC repository through Java Beans.

Go To Oxyus

BDDBot

DDBot is a web robot, search engine, and web server written entirely in Java. It was written as an example for a chapter on how to write your search engines, and as such it is very simplistic.

Go To BDDBot

Zilverline

Zilverline is what you could call a ‘Reverse Search Engine’. It indexes documents from your local disks (and UNC path style network disks), and allows you to search through them locally or if you’re away from your machine, through a webserver on your machine. Zilverline supports collections. A collection is a set of files and directories in a directory. PDF, Word, txt, java, CHM and HTML is supported, as well as zip and rar files. A collection can be indexed, and searched. The results of the search can be retrieved from local disk or remotely, if you run a webserver on your machine. Files inside zip, rar and chm files are extracted, indexed and can be cached. The cache can be mapped to sit behind your webserver as well.

Go To Zilverline

YaCy

This is a distributed web crawler and also a caching HTTP proxy. You are using the online-interface of the application. You can use this interface to configure your personal settings, proxy settings, access control and crawling properties. You can also use this interface to start crawls, send messages to other peers and monitor your index, cache status and crawling processes. Most important, you can use the search page to search either your own or the global index.

Go To YaCy

Compass

The Compass Framework is a first class open source Java framework, enabling the power of Search Engine semantics to your application stack declaratively. Built on top of the amazing Lucene Search Engine, Compass integrates seamlessly to popular development frameworks like Hibernate and Spring. It provides search capability to your application data model and synchronises changes with the datasource. With Compass: write less code, find data quicker.

Go To Compass

Lius

LIUS – Lucene Index Update and Search LIUS is an indexing Java framework based on the Jakarta Lucene project. The LIUS framework adds to Lucene many files format indexing fonctionalities as: Ms Word, Ms Excel, Ms PowerPoint, RTF, PDF, XML, HTML, TXT, Open Office suite and JavaBeans. LIUS is very easy to use; all the configuration of the indexing (types of files to be indexed, fields, etc…) as well as research is defined in a XML file, so the user only have to write few lines of code to carry out the indexing or research. LIUS has been developed from a range of Java technologies and full open source applications.

Go To Lius

Solr

Solr is an open source enterprise search server based on the Lucene Java search library, with XML/HTTP APIs, caching, replication, and a web administration interface.

Go To Solr

regain

´regain´ is a fast search engine on top of Jakarta-Lucene. It crawles through files or webpages using a plugin architecture of preparators for several file formats and data sources. Search requests are handled via browser based user interface using Java server pages. ´regain´ is released under LGPL and comes in two versions: 1. standalone desktop search program including crawler and http-server 2. server based installation providing full text searching functionality for a website or intranet fileserver using XML configuration files.

Go To regain

Piscator

Piscator is a small SQL/XML search engine. Once an XML feed is loaded, it can be queried using plain SQL. The setup is almost identical to the DB2 side tables approach.

Go To Piscator

MG4J

MG4J (Managing Gigabytes for Java) is a collaborative effort aimed at providing a free Java implementation of inverted-index compression techniques; as a by-product, it offers several general-purpose optimised classes, including fast and compact mutable strings, bit-level I/O, fast unsynchronised buffered streams, (possibly signed) minimal perfect hashing, etc. MG4J functions as a full-fledged text-indexing system. It can analyze, index, and query consistently large document collections.

Go To MG4J

Hounder

Hounder is a simple and complete search system. Out of the box, Hounder crawls the web targeting only those documents of interest, and presents them through a simple search web page and through an API, ideal for integrating into other projects. It is designed to scale on all fronts: the number of the indexed pages, the crawling speed and the number of simultaneous search queries. It is in use in many large scale search systems.

Go To Hounder

HSearch

HSearch is an open source, NoSQL Search Engine built on Hadoop and HBase. HSearch features include: * Multiple document formats * Record and document level search access control * Continuous index updating * Parallel indexing using multiple machines * Embeddable application * A REST-ful Web service gateway that supports XML * Auto sharding * Auto replication

Go To HSearch

Source

Best Open Source Search Engine Software for Enterprises

Many Enterprises have wishful thinking of having a search engine of their own. Fortunately, this no longer has to stay in a businessman’s imagination as it could be done through an open-source search engine software.

Having your own search engine lets you and your employees search for information within your enterprise domain. It also allows you to search data and information from intranets and multiple databases. In addition to that, you will also be able to work and save your own company’s important information.

An open-source search engine could be installed on a laptop first to test if the said software would suit your needs. Once you feel that it could accommodate everything that your company or enterprise’s needs, then you could already install it on your servers.

An open-source search engine works like the popular search engines, Google and Yahoo but it particularly caters startup companies, businesses and enterprises. Aside from it being able to act as a source of multiple databases and information, it could also index files, documents and emails as well from different file and document systems.

1. Apache Lucene Core

The Apache Lucene Core is probably the most popular open-source search engine software. It is known as one of the most reliable tool that is distributed by Apache License. The Apache Lucene Core is based and written on Java. Though this is the case, it is also available in other programming languages such as Ruby, Python, C++, C#, Perl, PHP and Delphi.

This search engine works on a ranking search system and pluggable ranking models such as Okapo BM25 and Vector Space Model. Also, aside from that, Apache Lucene Core supports numerous powerful query types such as wildcard, phrase, proximity, range queries and many more.

2. Elasticsearch Open-Source Search Engine

Elasticsearch is a RESTful search and analytics engine which is based on Apache Lucene. This open-source search engine is highly recommended to and is perfect use for small-medium to large scaled enterprises. It has the capacity for full text search and comes with a HTTP web interface. Aside from that, this open-source search engine offers a Schema-free JSON documents. It is also considered as a distributed search system that is fully sharded with configurable number of shards.

Elasticsearch is written in Java and is also available in different program languages such as PHP, Net (C#), Curl, Python Perl, Ruby, Apache Groovy and many more.

3. Apache Solr Search Engine Platform Open Source

Apache Solr Search Engine Platform Open Source is another great and popular open source search engine software option. It offers DB based ranking and is capable to accept full-text search and real-time indexing.

This open-source search engine software is based on Lucene and it uses Java search library. It could also do indexing in Solr through CSV, XML, JSON and binary through HTTP. In addition to that, this search engine software has capabilities to search both index and query.

4. Sphinx Search Engine

Sphinx Search Engine is a great Elasticsearch alternative and option. It is offered for free and is an open-source information retrieval software library and is capable to search full text. It also could work as a standalone server and it is written in C++ and could also work on Linux, MacOS, Windows, Solaris, FreeBSD and many more.

It could also search data stored in SQL database and NoSQL storage. It could also support highly documented websites and could index up to 10-15 MB of text per second per sing CPU core.

Key Features:

  • Non-text attributes support
  • Real-time full text indexes
  • SQL database indexing
  • Advanced full-text searching syntax
  • Easy application integration
  • Relevance ranking
  • Flexible text processing
  • Distributed searching

5. DataparkSearch Engine

DataparkSearch Engine is another amazing open source web-based search engine which allows its users to search within a website, intranet, local system and group of websites. In addition to that, this search engine features https, http, nntp and ftp. Aside from that, it also could support URL schemes and indexes text/html, audio/mpeg, text/plain and image/gif. It could also handle internationalized domain names.

6. Xapian

Xapian is an open source search engine which is written in C++ and could be used with other programming languages such as Python 2,3, Perl, PHP7,5, Tcl, Java, Ruby, C#, Erlang, Lua, R. and Node.js.

Source

14 Great Search Engines You Can Use Instead of Google


Google is a behemoth.

With its powerful algorithms, dominant advertising platform, and personalized user experience, Google is a force to be reckoned with – and one of the most popular search engines in the world.

That said, Google’s easy-to-use interface and personalized user experience comes at a cost.

It’s no secret the search engine giant catalogs the browsing habits of its users and shares that information with advertisers and other interested parties.

However, if you are unwilling to trade privacy for convenience, there are dozens of Google alternatives – many offering a better search experience.

Here are 14 search alternatives to Google.

Bing SearchBing Search

Despite trailing Google by a wide margin in U.S. market share (24.2 percent vs. 63.2 percent), an argument can be made that Bing performs better in certain aspects.

For starters, Bing has a rewards program that allows one to accumulate points while searching. These points are redeemable at the Microsoft and Windows stores, which is a nice perk.

The Bing image search performs flawlessly across all browsers, whereas Google image search seems to be optimized just for Chrome.

In my view, the Bing image search GUI is superior to its rival’s and much more intuitive. Bing carries that same clean user experience to video, making it the “go to” source for video search without a YouTube bias.

DuckDuckGo SearchDuckDuckGo Search

If you’re looking for true privacy, DuckDuckGo is the search engine for you.

DuckDuckGo doesn’t collect or store any of your personal information. That means you can run your searches in peace without having to worry about the boogeyman watching you through your computer screen.

DuckDuckGo is the perfect choice for those who wish to keep their browsing habits and personal information private.

Wiki.com SearchWiki.com Search

Looking for a search engine that pulls its results from thousands of wikis on the net? If so, Wiki.com is a good choice.

Wiki.com is the perfect search engine for those who appreciate community-led information as found on sites like Wikipedia.

Twitter searchTwitter search

Twitter is hard to beat as a real-time search engine. It’s the perfect place to go for minute by minute updates in the case of an emergency.

Google’s algorithm will catch up eventually, but nothing beats a Tweet in the heat of the moment.

Creative Commons SearchCreative Commons Search

CC Search should be your first stop on the hunt for many types of copyright-free content.

This search engine is perfect if you need music for a video, an image for a blog post, or anything else without worrying about an angry artist coming after you for ripping off their work.

The way CC Search works is simple – it draws in results from platforms such as Soundcloud, Wikimedia, and Flickr and displays results labeled as Creative Commons material.

Gibiru SearchGibiru Search

Are you wearing a MAGA hat while reading this? If so, Gibiru may be the search engine you’ve been looking for.

According to their website, “Gibiru is the preferred Search Engine for Patriots.”

They claim their Search results are sourced from a modified Google algorithm, so users are able to query the information they seek without worrying about Google’s tracking activities.

Because Gibiru doesn’t install tracking cookies on your computer they purport to be faster than “NSA Search Engines.”

Internet Archive SearchInternet Archive Search

 

The Wayback Machine is great for researching old websites, but it’s so much more.

As the name implies, this search engine queries a massive collection of documented material, including millions of free videos, books, music and software.

Essentially, Internet Archive is a vast online library where you can access just about anything you could imagine.

Search EncryptSearch Encrypt

Search Encrypt is a private search engine that uses local encryption to ensure your searches remain private.

It uses a combination of encryption methods that include Secure Sockets Layer encryption and AES-256 encryption.

When you input a query, Search Encrypt will pull the results from its network of search partners and deliver the requested information.

One of the best parts of Search Encrypt is that your search terms will eventually expire, so your information will remain private even if someone has local access to your computer.

Yandex searchYandex search

Looking for a search perspective outside of the United States?

Yandex is the most popular search engine in Russia, which is used by more than 53 percent of Russian Internet users. It is also used in Belarus, Kazakhstan, Turkey, and Ukraine.

Yandex is an overall easy-to-use search engine. As an added bonus, it offers a suite of some pretty cool tools.

For example, if you use its cloud storage service, Yandex Disk, you can search for your personal files right from the search bar of the search engine!

StartPage searchStartPage search

StartPage was developed to include results from Google, making it perfect for those who prefer Google’s search results without having to worry about their information being tracked and stored.

It also includes a URL generator, a proxy service, and HTTPS support. The URL generator is especially useful because it eliminates the need to collect cookies. Instead, it remembers your settings in a way that promotes privacy.

SwisscowsSwisscows

Swisscows is one of the more unique options on this list, billing itself as a family-friendly semantic search engine.

It uses artificial intelligence to determine the context of a user’s query. Over time, Swisscows promises to answer your questions with surprising accuracy.

Boardreader SearchBoardreader Search

If you’re interested in finding a forum or message board about a specific subject, Boardreader should be the first place you turn to.

This search engine queries its results from a wide variety of message boards and forums online. You should be able to find the forum you want with just a few keystrokes.

SlideShare SearchSlideShare Search

This unique search engine allows you to search for documented slideshow presentations.

You can also search for ebooks and PDFs, making it an excellent tool if you have a business presentation to prepare for.

SlideShare also allows you to save slides and even download the entire slideshow for use on your local computer.

Ecosia SearchEcosia Search

Looking to save the planet, one tree at a time? Then check out this environmentally friendly search engine!

This may come as a surprise, but your Google searches actually contribute to the creation of quite a bit of CO2.

To battle this issue, Ecosia uses the revenues generated from search engine queries to plant trees. Typically Ecosia needs around 45 searches to plant a new tree.

Bottom Line

Google may be the most popular choice in search engines, but you still have a multitude of alternatives to use.

Many of these alternative search engines provide a better user experience and superior information to Google.

Whether you’re looking for more privacy or simply want to explore your options, there are plenty of search engines to experiment with. So what are you waiting for?

Image Credits

Featured Image: geralt / Pixabay
All screenshots taken by author, September 2018

Category

Source

11 Best Privacy Oriented Search Engines To Google in 2020

Brief: In this age of the internet, you can never be too careful with your privacy. Use these alternative search engines that do not track you.

Best privacy oriented alternative search enginesBest privacy oriented alternative search engines

Google – unquestionably being the best search engine out there, makes use of powerful and intelligent algorithms (including A.I. implementations) to let the users get the best out of a search engine with a personalized experience.

This sounds good until you start to live in a filter bubble. When you start seeing everything that ‘suits your taste’, you get detached from reality. Too much of anything is not good. Too much of personalization is harmful as well.

This is why one should get out of this filter bubble and see the world as it is. But how do you do that?

You know that Google sure as hell tracks a lot of information about your connection and the system when you perform a search and take an action within the search engine or use other Google services such as Gmail.

So, if Google keeps on tracking you, the simple answer would be to stop using Google for searching the web. But what would you use in place of Google? Microsoft’s Bing is no saint either.

So, to address the netizens concerned about their privacy while using a search engine, I have curated a list of privacy oriented alternative search engines to Google. 

Best 8 Privacy-Oriented Alternative Search Engines To Google

Do note that the alternatives mentioned in this article are not necessarily “better” than Google, but only focuses on protecting users privacy. Here we go!

1. DuckDuckGo

Duckduckgo Dark ModeDuckduckgo Dark ModeDuckDuckGo (Dark Mode)

DuckDuckGo is one of the most successful privacy-oriented search engines that stands as an alternative to Google. The user experience offered by DuckDuckGo is commendable. I must say – “It’s unique in itself”.

DuckDuckGo, unlike Google, utilizes the traditional method of “sponsored links” to display the advertisements. The ads are not focused on you but only the topic you are searching for – so there is nothing that could generate a profile of you in any manner – thereby respecting your privacy.

Of course, DuckDuckGo’s search algorithm may not be the smartest around (because it has no idea who you are!). And, if you want to utilize one of the best privacy oriented alternative search engines to Google, you will have to forget about getting a personalized experience while searching for something.

The search results are simplified with specific meta data. It lets you select a country to get the most relevant result you may be looking for. Also, when you type in a question or search for a fix, it might present you with an instant answer (fetched from the source).

Although, you might miss quite a few functionalities (like filtering images by license) – that is an obvious trade-off to protect your privacy.

DuckDuckGo

2. Qwant

best privacy oriented search enginebest privacy oriented search engine

Qwant is probably one of the most loved privacy oriented search engines after DuckDuckGo. It ensures neutrality, privacy, and digital freedom while you search for something on the Internet.

If you thought privacy-oriented search engines generally tend to offer a very casual user experience, you need to rethink after trying out Qwant. This is a very dynamic search engine with trending topics and news stories organized very well. It may not offer a personalized experience (given that it does not track you) – but it does offer a rich user experience.

Qwant is a very useful search engine alternative to Google. It lists out all the web resources, social feeds, news, and images on the topic you search for.

Qwant

3. Startpage

Startpage ScreenshotStartpage Screenshot

Attention!

System1 has recently acquired Startpage. While being a digital advertising company System1 claims that they are providing privacy focused products. Hence, we cannot vouch for their claim.

In other words, it’s up to you to trust System1 and Startpage.

Startpage is a good initiative as a privacy-oriented search engine alternative to Google. However, it may not be the best one around. The UI is very similar to that of Google’s (while displaying the search results – irrespective of the functionalities offered). It may not be a complete rip-off but it is not very impressive – everyone has got their own taste.

To protect your privacy, it lets you choose it. You can either select to visit the web pages using the proxy or without it. It’s all your choice. You also get to change the theme of the search engine. Well, I did enjoy my switch to the “Night” theme. There’s an interesting option with the help of which you can generate a custom URL keeping your settings intact as well.

Startpage

4. Privatelee (Discontinued)

best privacy oriented search enginebest privacy oriented search engine

Privatelee was another kind of search engine specifically tailored to protect your online privacy. It did not track your search results or behavior in any way. However, you used to get a lot of irrelevant results after the first ten matched results.

The search engine wasn’t perfect to find a hidden treasure on the Internet but more for general queries. Privatelee also supported power commands – more like shortcuts – which helps you search for the exact thing in an efficient manner. It would save a lot of your time for pretty simple tasks such as searching for a movie on Netflix. If you were looking for a super fast privacy oriented search engine for common queries, Privatelee would have been a good alternative to Google.

Privatelee

5. Swisscows

best privacy oriented search enginebest privacy oriented search engine

Well, it isn’t a dairy farm portfolio site but a privacy-oriented search engine as an alternative to Google. You may have known about it as Hulbee– but it has recently redirected its operation to a new domain. Nothing has really changed except for the name and domain of the search engine. It works the same way it was before as Hulbee.com.

Swisscows utilizes Bing to deliver the search results as per your query. When you search for something, you would notice a tag cloud on the left sidebar which is useful if you need to know about the related key terms and facts.

The design language is a lot simpler but one of its kind among the other search engines out there. You get to filter the results according to the date but that’s about it – no more advanced options to tweak your search results. It utilizes a tile search technique (a semantic technology) to fetch the best results for your queries. The search algorithm makes sure that it is a family-friendly search engine with pornography and violence ruled out completely.

Swisscows

6. searX

best privacy oriented search enginebest privacy oriented search engine

searX is an interesting search engine – which is technically defined as a “metasearch engine”. In other words, it utilizes other search engines and accumulates the results to your query in one place. It does not store your search data being an open source metasearch engine at the same time. You can review the source code, contribute, or even customize it as your own metasearch engine hosted on your server.

If you are fond of utilizing Torrent clients to download stuff, this search engine will help you find the magnet links to the exact files when you try searching for a file through searX. When you access the settings (preferences) for searX, you would find a lot of advanced things to tweak from your end.

General tweaks include – adding/removing search engines, rewrite HTTP to HTTPS, remove tracker arguments from URL, and so on. It’s all yours to control. The user experience may not be the best here but if you want to utilize multiple search engines while keeping your privacy in check, searX is a great alternative to Google.

The only problem here is, you may not have a single domain active for the search engine. Hence, there are multiple searx instances, if the button below does not work, you should browse the list of instances to look for others or just host it yourself.

searX

7. Peekier

best privacy oriented search enginebest privacy oriented search engine

Peekier is another fascinating privacy oriented search engine. Unlike the previous one, it is not a metasearch engine but has its own algorithm implemented. It may not be the fastest search engine I’ve ever used but it is an interesting take on how search engines can evolve in the near future. When you type in a search query, it not only fetches a list of results but also displays the preview images of the web pages listed. So, you get a “peek” on what you seek. While the search engine does not store your data, the web portals you visit do track you.

So, in order to avoid that to an extent, Peekier accesses the site and generates a preview image to decide whether to head into the site or not (without you requiring to access it). In that way, you allow fewer websites to know about you – mostly the ones you trust.

Peekier

8. MetaGer

best privacy oriented search enginebest privacy oriented search engine

MetaGer is yet another open source metasearch engine. However, unlike others, it takes privacy more seriously and enforces the use of Tor network for anonymous access to search results from a variety of search engines. Some search engines who claim to protect your privacy may share your information to the government (whatever they record) because the server is bound to US legal procedures. However, with MetaGer, the Germany-based server would protect even the anonymous data recorded while using MetaGer.

They do house a few number of advertisements (without trackers of course)- but you can get rid of those as well by joining in as a member of the non-profit organization – SUMA-EV – which sponsors the MetaGer search engine.

Metager

9. Ecosia

EcosiaEcosia

I used Ecosia for a while as my primary search engine. It’s a one-of-a-kind privacy-focused search engine that actually plants trees if you use it.

They utilize Bing’s search results at the core – however, I didn’t observe any trackers while using it. When you use their search engine, they make money (which includes monetary benefits from sponsored ads). Next, they contribute a significant amount of money to notable organizations and activists helping plant more trees.

At first, this might seem to be controversial. But, they share monthly financial reports and I’ve also observed respectable organizations involved with Ecosia to help plant more trees. In addition to all this, they claim that their servers run on 100% renewable energy.

Ecosia

10. Gibiru

Gibiru Search EngineGibiru Search Engine

Gibiru is a privacy-friendly search engine that aims for uncensored search results. It doesn’t enforce any trackers but it recommends you to utilize ExpressVPN in addition to their search engine service, in order to prevent other websites to track your activity.

The search results may not be the best around – but it puts some interesting uncensored search results. You should give it a try.

Gibiru

11. Mojeek

MojeekMojeek

Mojeek has been around for a long time now. They’re an independent ‘crawler-based’ search engine, based in the UK, with their own algorithm and index of web pages.

If you are looking for a privacy-focused search engine that does not enforce any trackers while having its own index for search results, you should be good to go. I tried searching for some common queries and was satisfied with the search results. I think you can give it a try for yourself.

Mojeek

Wrapping Up

If you are concerned about your privacy, you should also take a look at some of the best privacy-focused Linux distributions. Among the search engine alternatives mentioned here – DuckDuckGo – is my personal favorite. But it really comes down to your preference and whom would you choose to trust while surfing the Internet.

Do you know some more interesting (but good) privacy-oriented alternative search engines to Google?

Let us know your thoughts in the comments below.

515 Shares

Source

What are the free/open source search engines?

Searx is probably the most popular open source search engine and it has a lot of great features with it. There’s also YaCy which is a decentralized P2P search engine, though it probably can’t be used as a daily driver it’s definitely important that it exists.

DuckDuckGo does have some open source code but it’s not entirely open, so whether or not it qualifies enough for you is subjective.

There may be other odd balls like Gigablast but generally speaking they’re all either out-dated or simply not a popular choice for one reason or another.

Source

Sourcegraph: An Open-Source Source Code Search Engine

Last year, the Code Search and Navigation tool ‘Sourcegraph‘ was declared Open Source. As it makes navigating through Source Code much more convenient, the tool itself going Open Source was definitely a big plus for developers!

We’ve looked into its features and also tried to find out how it can be so helpful for developers who are used to navigate through code hosts like GitHub, GitLab and others quite regularly.

Now, with its new 3.10 release, let us take a look at how it helps and what’s new.

Sourcegraph Features

As stated on their GitHub page, Sourcegraph has the following features:

  • Fast global Code search
  • Intelligent Code recognition
  • Code host Enhancement on GitHub, GitLab and more
  • Extension API for easier third-party integration

New Features in Sourcegraph 3.10

You will find the latest Sourcegraph 3.10 added with significant improvements and features.

The key highlights for the latest release as mentioned in their official blog post are:

Improved search autocompletion, native GitLab integration, and search and replace automation campaigns

Let’s take a quick look at what has changed with Sourcegraph 3.10:

  • Sourcegraph now provides native code intelligence to GitLab
  • Improved autocompletion for search query filters
  • The ability to create cross-repository search and replace campaigns
  • LSIF-based precise code intelligence now supports five languages: Go, Typescript, Java, C++, and Python
  • Fully automated release testing process

For more information, you can view the complete changelog.

You can deploy Sourcegraph on your server and configure it to work with your or your organization’s Git repositories. Once that’s done, you get a search engine where you can search all the codes.

But if you are a lone developer, like me, you can still use Sourcegraph on GitHub or GitHub alternatives like GitLab

I am going to quickly show you how to use Sourcegraph for better code navigation on GitHub.

Using Sourcegraph on GitHub

Let’s find out how you can easily try this tool with a Firefox or Chrome extension. Here is the official extensions page.

Sourcegraph Official ExtensionsSourcegraph Official ExtensionsSourcegraph Official Extensions

Security Issue Open on GitHub

During the first half of June, I reported a security issue to Sourcegraph that is now currently open on GitHub. Apparently, the issue was fixed around a week ago but yet to be merged to the master branch. The issue is about the official Firefox extension that asks for access to all websites during installation while there should be a filter for repository sites only such as GitHub, GitLab, BitBucket, and others. Originally, due to a Firefox bug (now fixed), it could not be incorporated during early development.

This is how it looks like with the official Sourcegraph extension installed and when you view a file on the Vim repository on GitHub:

Sourcegraph source code engineSourcegraph source code engineSourcegraph extension on GitHubSourcegraph extension on GitHub

Note how we can see the new Sourcegraph buttons within the GitHub interface, thanks to the installed extension. One thing to note is that one need not even login into GitHub to navigate through hosted Code and their repositories in order to make use of the helpful features of Sourcegraph.

When you click on “View File”, the entire look changes and the file is opened for you in a completely new interface within the browser itself:

Sourcegraph extension on GitHubSourcegraph extension on GitHub

Without Sourcegraph, if you want to look for files with a particular format, say C++ .cpp files in this example, it is very difficult to filter and view them if we try to use GitHub’s own search engine within this repository:

Sourcegraph extension on GitHubSourcegraph extension on GitHub

But once you are using this extension, see how easily you can view all such files in one go within the repository:

Sourcegraph extension on GitHubSourcegraph extension on GitHubSourcegraph extension on GitHubSourcegraph extension on GitHub

Sourcegraph can narrow down through Code Search very intelligently as explained in this video:

Code intelligence in Sourcegraph is powered by Lang Server, which enables identifying the type of Programming Language you are using:

Language support SourcegraphLanguage support Sourcegraph

Learn more about its usefulness in the following video:

Bonus Tip on using Sourcegraph 

Even without installing an extension on your browser, you can directly use Sourcegraph as an IDE on top of any repository on GitHub by just adding “sourcegraph.com/” as a prefix to the repository URL.

For example, the URL for the official Vim repository is:

github.com/vim/vim

To view the same through Sourcegraph, modify the URL as below and you’re good to go:

sourcegraph.com/github.com/vim/vim

I’ve also tested this method with GitLab and it works there too! You can try other repositories as well! There is an unofficial extension called Open on Sourcegraph that uses this method on Firefox and Chrome.

Sourcegraph Developers have a master plan behind declaring it Open Source:

Make basic code intelligence ubiquitous (for every language, and in every editor, code host, etc.)
Make code review continuous and intelligent
Increase the amount and quality of open-source code

Here are the ways they suggest you can contribute to its Development:

So this was a brief look into how Sourcegraph can make the developer’s life a lot more easier and hassle-free. You may also want to take a look at Sourcetrail, a recently open source project that allows you to visualize code base.

Are you a Developer? Would you like to adopt this new Open Source tool in your day-to-day programming tasks? Let us know in the comments section below.

95 Shares

Source