This article demonstrates a significantly different approach to managing probative risks arising from the complex and fast changing relationship between law and computer science. Law's historical problem in adapting to scientific and technologically dependent evidence production is seen less as a socio-techno issue than an ethical failure within criminal justice. This often arises because of an acceptance of epistemological incomprehension between lawyers and scientists. Something compounded by the political economy of criminal justice and safeguard evasion within state institutions. What is required is an exceptionally broad interdisciplinary collaboration to enable criminal justice decision-makers to understand and manage the risk of further ethical failure. If academic studies of law and technology are to address practitioner concerns, it is often necessary, however, to step down the doctrinal analysis to a specific jurisdictional level.

The transformation of the contemporary societies through digital technologies has had a profound effect on all human activities including those that are in the realm of illegal, unlawful, and criminal deeds. Moreover, the affordances provided by the anonymity creating techniques such as the Tor protocol which are beneficial for preserving civil liberties, appear to be highly profitable for various types of miscreants whose crimes range from human trafficking, arms trading, and child pornography to selling controlled substances and racketeering. The Tor similar technologies are the foundation of a vast, often mysterious, sometimes anecdotal, and occasionally dangerous space termed as the Dark Web. Using the features that make the Internet a uniquely generative knowledge agglomeration, with no borders, and permeating different jurisdictions, the Dark Web is a source of perpetual challenges for both national and international law enforcement agencies. The anonymity granted to the wrong people increases the complexity and the cost of identifying both the crimes and the criminals, which is often exacerbated with lack of proper human resources. Technologies such as machine learning and artificial intelligence come to the rescue through automation, intensive data harvesting, and analysis built into various types of web crawlers to explore and identify dark markets and the people behind them. It is essential for an effective and efficient crawling to have a pool of dark sites or onion URLs. The research study presents a way to build a crawling mechanism by extracting onion URLs from malicious executables by running them in a sandbox environment and then analysing the log file using machine learning algorithms. By discerning between the malware that uses the Tor network and the one that does not, we were able to classify the Tor using malware with an accuracy rate of 91% with a logistic regression algorithm. The initial results suggest that it is possible to use this machine learning approach to diagnose new malicious servers on the Tor network. Embedding this kind of mechanism into the crawler may also induce predictability, and thus efficiency in recognising dark market activities, and consequently, their closure. 

Strong encryption algorithms and reliable anonymity routing have made cybercrime investigation more challenging. Hence, one option for law enforcement agencies (LEAs) is to search through unencrypted content on the Internet or anonymous communication networks (ACNs). The capability of automatically harvesting web content from web servers enables LEAs to collect and preserve data prone to serve as potential leads, clues, or evidence in an investigation. Although scientific studies have explored the field of web crawling soon after the inception of the web, few research studies have thoroughly scrutinised web crawling on the “dark web”, or ACNs, such as I2P, IPFS, Freenet, and Tor. The current paper presents a systematic literature review (SLR) that examines the prevalence and characteristics of dark web crawlers. From a selection of 58 peer-reviewed articles mentioning crawling and the dark web, 34 remained after excluding irrelevant articles. The literature review showed that most dark web crawlers were programmed in Python, using either Selenium or Scrapy as the web scraping library. The knowledge gathered from the systematic literature review was used to develop a Tor-based web crawling model into an already existing software toolset customised for ACN-based investigations. Finally, the performance of the model was examined through a set of experiments. The results indicate that the developed crawler was successful in scraping web content from both clear and dark web pages, and scraping dark marketplaces on the Tor network. The scientific contribution of this paper entails novel knowledge concerning ACN-based web crawlers. Furthermore, it presents a model for crawling and scraping clear and dark websites for the purpose of digital investigations. The conclusions include practical implications of dark web content retrieval and archival, such as investigation clues and evidence, and related future research topics.

In the last decade, the proliferation of machine learning (ML) algorithms and their application on big data sets have benefited many researchers and practitioners in different scientific areas. Consequently, the research in cybercrime and digital forensics has relied on ML techniques and methods for analyzing large quantities of data such as text, graphics, images, videos, and network traffic scans to support criminal investigations. Complete and accurate training data sets are indispensable for efficient and effective machine learning models. An essential part of creating complete and accurate data sets is annotating or labelling data. We present a method for law enforcement agency investigators to annotate and store specific dark web content. Using a design science strategy, we design and develop tools to enable and extend web content annotation. The annotation tool was implemented as a plugin for the Tor browser. It can store web content, thus automatically creating a dataset of dark web data pertinent to criminal investigations. Combined with a central storage management server, enabling annotation sharing and collaboration, and a web scraping program, the dataset becomes multifold, dynamic, and extensive while maintaining the forensic soundness of the data saved and transmitted. To manifest our toolset's fitness of purpose, we used our dataset as training data for ML based classification models. A five cross-fold validation technique was used to evaluate the classifiers, which reported an accuracy score of 85 - 96%. In the concluding sections, we discuss the possible use-cases of the proposed method in real-life cybercrime investigations, along with ethical concerns and future extensions.

The generative and transformative nature of the Internet which has become a synonym for the infrastructure of the contemporary digital society, is also a place where there are unsavoury and illegal activities such as fraud, human trafficking, exchange of control substances, arms smuggling, extremism, and terrorism. The legitimate concerns such as anonymity and privacy are used for proliferation of nefarious deeds in parts of the Internet termed as a deep web and a dark web. The cryptographic and anonymity mechanisms employed by the dark web miscreants create serious problems for the law enforcement agencies and other legal institutions to monitor, control, investigate, prosecute, and prevent the range of criminal events which should not be part of the Internet, and the human society in general. The paper describes the research on developing a framework for identifying, collecting, analysing, and reporting information from the dark web in a forensically sound manner. The framework should provide the fundamentals for creating a real-life system that could be used as a tool by law enforcement institutions, digital forensics researchers and practitioners to explore and study illicit actions and their consequences on the dark web. The design science paradigms is used to develop the framework, while international security and forensic experts are behind the ex-ante evaluation of the basic components and their functionality, the architecture, and the organization of the system. Finally, we discuss the future work concerning the implementation of the framework along with the inducement of some intelligent modules that should empower the tool with adaptability, effectiveness, and efficiency.