OSINT – Website files metadata

Open-source intelligence (OSINT) is intelligence collected from publicly available sources. In the intelligence community (IC), the term “open” refers to overt, publicly available sources (as opposed to covert or clandestine sources); it is not related to open-source software or public intelligence.
OSINT includes all publicly accessible sources of information, such as:
– Media
– Web-based communities and user-generated content
– Observation and reporting
– Professional and academic (including grey literature)
– Deep Web – Information hidden from the Surface web currently estimated to represent the majority of content on the Web
OSINT is distinguished from research in that it applies the process of intelligence to create tailored knowledge supportive of a specific decision by a specific individual or group.
— wikipedia

• passive information gathering
• twitter search engine
• knowem
• twoogel
• keotag
• whostalkin
• pixsy
• flickr

Google dorks to search in Facebook
– Group Search: site:facebook.com inurl:group (bofa | “bank of america”)
– Group Wall Posts Search: site:facebook.com inurl:wall (bofa | “bank of america”)
– Pages Search: site:facebook.com inurl:pages (bofa | “bank of america”)
– Public Profiles: allinurl: people “John Doe” site:facebook.com
Google dorks to search in MySpace
– Profiles: site:myspace.com inurl:profile (bofa | “bank of america”)
– Blogs: site:myspace.com inurl:blogs (bofa | “bank of america”)
– Videos: site:myspace.com inurl:vids (bofa | “bank of america”)
– Jobs: site:myspace.com inurl:jobs (bofa | “bank of america”)
Google dorks to search in LinkedIn
– Public Profiles: site:linkedin.com inurl:pub (bofa | “bank of america”)
– Updated Profiles: site:linkedin.com inurl:updates (bofa | “bank of america”)
– Company Profiles: site:linkedin.com inurl:companies (bofa | “bank of america”)
You can easily modify the above dorks to search in other social networks or include more advanced search operators. With most social networks if you want to find private information you need to login as a user.
OSINT includes all publicly accessible sources of information, such as:
– Media
– Web-based communities and user-generated content
– Observation and reporting
– Professional and academic (including grey literature)
– Deep Web – Information hidden from the Surface web currently estimated to represent the majority of content on the Web
OSINT is distinguished from research in that it applies the process of intelligence to create tailored knowledge supportive of a specific decision by a specific individual or group.
— wikipedia
Gather information and documents

• OSINT part 1
• Icerocket
• Yahoo groups
• Google groups
• Boardreader
• Craigslist forums
• Vault
• Google finance
• Scribd
• Slideshare
• Pdf search

Extract documents metadata

• exiftool
• metagoofil
• metadata-extractor
• Information gathering types
  Passive
  During passive information gathering you should never send any type of traffic directly to the target. Passive I.G. allows the greatest amount of anonymity.
  Active
  During active information gathering you are sending requests to remote services and receiving responses based on the service type. This method includes, but is not limited to: DNS zone transfers, DNS reverse lookup, SMTP querying, SNMP enumeration, DNS bruteforcing, banner grabbing and smtp bruteforcing.
  Semi-passive
  During semi-passive information gathering you generate, what would be considered, normal traffic. You may contact the target but the requests need to look like all of the traffic is being generated from normal requests.
  OSINT Part 2
  securityblog.gr
  Professional and business social networks
  
• Zoominfo
• Xing
• Linkedin
• Pipl
• Meettheboss
• Spoke
• searchbug
• entitycube
• EDGAR

People information

• Pipl
• Spokeo
• 123people
• Yasni
• Knowem
• Namechk
• Yammer
• zabasearch
• searchbug
• skipease
• addictomatic
• entitycube
• EDGAR
• peekyou

Image search

• Tineye

•  
  OSINT – Website files metadata
  Posted on 05/01/2017, 09:57 By Johnny Cash
  Download Website files
  

  wget -r -l2 http://example.com

  Options
  

  -r retrieve files recursively -l2 maximum depth

  
  Download exiftool
  
  ExifTool is a platform-independent Perl library plus a command-line application for reading, writing and editing meta information in a wide variety of files. ExifTool supports many different metadata formats.
  http://www.sno.phy.queensu.ca/~phil/exiftool/
  
  Extract files metadata
  

  /path/to/exiftool -r -h -a -u -g1 * > metadata-report.html

  Options
  

  -r recursively process subdirectories
  -h use HMTL formatting for output
  -a allow duplicate tags to be extracted
  -u extract unknown tags
  -g1 organize output by tag group
  * all files

  1. Nmap
  Nmap (“Network Mapper”) is a free and open source (license) utility for network discovery and security auditing. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running, what type of packet filters/firewalls are in use, and dozens of other characteristics. Nmap homepage.
  
  2. Wireshark
  Wireshark is a network protocol analyzer. It lets you capture and interactively browse the traffic running on a computer network. Wireshark homepage.
  
  3. Metasploit Community edition
  Metasploit Community Edition simplifies network discovery and vulnerability verification for specific exploits, increasing the effectiveness of vulnerability scanners. This helps prioritize remediation and eliminate false positives, providing true security risk intelligence. Metasploit community edition homepage.
  
  4. Nikto2
  Nikto is an Open Source (GPL) web server scanner which performs comprehensive tests against web servers for multiple items, including over 6400 potentially dangerous files/CGIs, checks for outdated versions of over 1200 servers, and version specific problems on over 270 servers. It also checks for server configuration items such as the presence of multiple index files, HTTP server options, and will attempt to identify installed web servers and software. Nikto2 homepage.
  
  5. John the Ripper
  John the Ripper is a fast password cracker, currently available for many flavors of Unix, Windows, DOS, BeOS, and OpenVMS. Its primary purpose is to detect weak Unix passwords. Besides several crypt(3) password hash types most commonly found on various Unix systems, supported out of the box are Windows LM hashes, plus lots of other hashes and ciphers in the community-enhanced version. John the Ripper homepage.
  
  6. ettercap
  Ettercap is a comprehensive suite for man in the middle attacks. It features sniffing of live connections, content filtering on the fly and many other interesting tricks. It supports active and passive dissection of many protocols and includes many features for network and host analysis. ettercap homepage.
  
  7. NexPose Community edition
  The Nexpose Community Edition is a free, single-user vulnerability management solution. Nexpose Community Edition is powered by the same scan engine as Nexpose Enterprise and offers many of the same features. Nexpose homepage.
  
  8. Ncat
  Ncat is a feature-packed networking utility which reads and writes data across networks from the command line. Ncat was written for the Nmap Project as a much-improved reimplementation of the venerable Netcat. It uses both TCP and UDP for communication and is designed to be a reliable back-end tool to instantly provide network connectivity to other applications and users. Ncat will not only work with IPv4 and IPv6 but provides the user with a virtually limitless number of potential uses. ncat homepage.
  
  9. Kismet
  Kismet is an 802.11 layer2 wireless network detector, sniffer, and intrusion detection system. Kismet will work with any wireless card which supports raw monitoring (rfmon) mode, and (with appropriate hardware) can sniff 802.11b, 802.11a, 802.11g, and 802.11n traffic. Kismet also supports plugins which allow sniffing other media such as DECT. kismet homepage.
  
  10. w3af
  w3af is a Web Application Attack and Audit Framework. The project’s goal is to create a framework to find and exploit web application vulnerabilities that is easy to use and extend. w3af homepage.
  
  11. hping
  hping is a command-line oriented TCP/IP packet assembler/analyzer. The interface is inspired to the ping(8) unix command, but hping isn’t only able to send ICMP echo requests. It supports TCP, UDP, ICMP and RAW-IP protocols, has a traceroute mode, the ability to send files between a covered channel, and many other features. hping homepage.
  
  12. burpsuite
  Burp Suite is an integrated platform for performing security testing of web applications. Its various tools work seamlessly together to support the entire testing process, from initial mapping and analysis of an application’s attack surface, through to finding and exploiting security vulnerabilities. BurpSuite homepage.
  
  13. THC-Hydra
  A very fast network logon cracker which support many different services.  hydra homepage.
  
  14. sqlmap
  sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester and a broad range of switches lasting from database fingerprinting, over data fetching from the database, to accessing the underlying file system and executing commands on the operating system via out-of-band connections. sqlmap homepage.
  
  15. webscarab
  WebScarab has a large amount of functionality, and as such can be quite intimidating to the new user. But, for the simplest case, intercepting and modifying requests and responses between a browser and HTTP/S server, there is not a lot that needs to be learned. WebScarab homepage.
  

http://securityblog.gr/4046/osint-part-1/