i Search engine for security researches ( censys.io) – All things in moderation

Search engine for security researches ( censys.io)

1.Introduction about censys.io

censys.io_interface

Censys is a search engine that allows computer scientists to ask questions about the devices and networks that compose the Internet. Driven by Internet-wide scanning, Censys lets researchers find specific hosts and create aggregate reports on how devices, websites, and certificates are configured and deployed.

If you simply search for a word or phrase, Censys will return any records that contain the phrase. For example, searching for nginx will return any records that contain the word nginx. Searching for 23.0.0.0/8 will return all hosts in that network. Check out some of our example searches.

simple_search_censysio

2.Features and architecture of censys.io

 Censys System Architecture

Censys is driven by application scans of the IPv4 address space, which are scheduled onto a pool of scan workers. These workers complete scans, extract valuable fields, and annotate records with additional metadata in order to generate structured data about each host. These records are centrally managed in a custom database engine, ZDb, which maintains the current state of every host. ZDb feeds updated records to a web front-end where researchers can query the data.

protocol_scanning_and_annotation

Each scan worker uses Zmap to perform host discovery for a shard of the IPv4 address, and completes protocol handshakes using pluggable application scanners. Censys extracts fields of interest and annotates records with additional metadata. The information from a protocol handshake is converted to an atom – a deterministic data structure describing a specific protocol on a host.

3.Search Syntax

  • Specifying Fields

Censys records are structured and allow querying specific fields. For example, you can search for all hosts with a specific HTTP status code with the following query: 80.http.get.status_code: 200. You can view a list of defined fields under the Data Definitions tab or by looking at the details of a host. For example, here are the fields for the Censys web server.

  • Boolean Logic

You can compose multiple statements using the terms and, or, not, and parentheses. For example, (“Schneider Electric” or Dell) and 23.20.0.0/14. By default, all included terms are optional (i.e., executed as an or statement).

  • Networks, Host Names, and Protocols

You can search for IP addresses using CIDR notation (e.g., ip:23.20.0.0/14) or by specifying a range of addresses: ip:[23.20.0.0 TO 23.20.5.34]. You can search for hosts that serve a particular protocol by searching the protocols field, e.g., protocols: “102/s7”.

Inline DNS queries are possible with the following syntax: a:facebook.com and mx:gmail.com.

  • Ranges

You can search for ranges of numbers using [ and ] for inclusive ranges and { and } for exclusive ranges. For example, 80.http.get.status_code:[200 TO 300]. Dates should be formatted using the following syntax: [2012-01-01 TO 2012-12-31]. One sided limits can also be specified: [2012-01-01 TO *]. Warning! The TO operator must be capitalized.
Wildcards and Regular Expressions

By default, Censys searches for complete words. In other words, the search Del will not return records that contain the word Dell. Wildcard searches can be run on individual terms, using ? to replace a single character, and * to replace zero or more characters. For example, if you want to search for words that start with Del, you would search for Del*.

You can also search using regular expressions, e.g., metadata.manufacturer:/De[ll]/. The full regex syntax is available here.

  • Boosting

The boost operator (^) can be used to make one term more relevant than another. For example, metadata.manufacturer:__ Dell^2 OR “Schneider Electric”__ places more preference on the Dell keyword.

  • Reserved Characters

The following characters must be escaped with a backslash: + – = && || > < ! ( ) { } [ ] ^ ” ~ * ? : \ /.

Example search :
example_search_censysio.

I think it’s enough for this post, in the next post I will explain about exposing data and applications . :)))

Leave a Reply