Like the Light, the Darkness is a cosmic paracausal force that has existed since before the beginning of time. In our universe, it has been shown to enact its will indirectly through numerous proxies and servants, with some of the most powerful being worshiped as deities unto themselves. The Vex, Hive, Taken and Scorn are all linked to this force.
The Darkness is fundamentally driven to reduce complexity and diversity within the universe, and to eliminate any entity which cannot survive in the face of adversity; its philosophy is tied to that of the Sword Logic, though not necessarily identical to it. The Darkness once ruled the universe, and the Hive believe that the Light must be consumed for the Darkness to reclaim its domination. Its ultimate goal is to bring about an end-state of the universe in which the only entities that exist are those that have overcome all possible obstacles to their survival, and which have the strength to continue existing forever. It refers to this goal as the "final shape" of the universe.
The Awoken are said to carry Darkness within themselves as well as Light, a product of the circumstances of their creation. During his possession by the Taken Ahamkara known as Riven, Uldren Sov emitted streams of Darkness from his eyes to activate the gate of the Dreaming City. The existence of Motes of Dark further indicates that the Darkness is a universal force in a similar sense as the Light.
The Darkness seemingly also has the ability to 'infect' individuals. Though it is unknown if Uldren's infection was caused by fallout from the Dreadnaught weapon or Riven's influence, he began to display symptoms on Mars (in his unique case, irritation of the eyes), and gradually descended into mood swings, bouts of lethargy, extreme sensitivity to Ahamkara influence, bloodthirsty tendencies, and almost completely forgetting, and turning against those he once considered close friends (Jolyon and Petra, respectively). In a similar case, Dredgen Yor fell due to being heavily seduced by the Darkness.
Based upon matching descriptions from Emperor Calus, the Hive, Cayde-6, and Awoken records, the Darkness and certain Darkness-related entities seems to have the ability to project some sort of field that gives the impression of complete darkness surrounding an individual. Calus wrote that upon encountering the Darkness, it appeared as a complete void to him, while the Awoken reported being surrounded by a force that "made the stars go out." Cayde-6 similarly described being enveloped by a smothering, paralyzing black cloud during the Collapse; however, in this case it left his senses intact so that he could witness the destruction of his surroundings. The Drifter also encountered mysterious entities on an unknown, distant ice world that were trapped in alien monoliths. These beings had a disturbing ability to kill Ghosts from just being near them, and to nullify Light itself. The beings were everywhere on the world, and were described as having a "Darkness field."
In addition to being a fundamental paracausal force, The Darkness is intelligent, and capable of communicating with sentient beings; in the Books of Sorrow, Oryx is described as directly communing with the Deep, and the deposed Cabal Emperor Calus is recorded as doing so as well. In Shadowkeep, the Darkness communicates with The Guardian through their possessed Ghost, describing the Light as a source of weakness, failure and death for all those it touches.
Pyramids and Veiled Figures
By Destiny 2, many depictions and allusions to the Darkness now represent it as an armada of Pyramids located in the depths of intergalactic space. These pyramidal entities were first seen in the vision that the Traveler sent to the the Guardian, and also in the cutscene that showed the Traveler being attacked on Io. The Dreams of Alpha Lupi describe the Darkness as a "knife [with] a million blades," perhaps alluding to this fleet of pyramidal entities. During the Collapse, the human colonists who would become the Awoken detected that they were being followed by a "phantom ship" that exposed them to gravitational waves and emitted sterile neutrinos, axions, and phaetons. During his encounter with the Darkness, Calus also reported the "nothingness" around him parting to reveal a fleet of ships, although his statements do not describe the vessels in detail.
Alternatively, the Pyramids are noted to be entities entirely analogous to the Traveler, which is an embodiment of the Light but not the source of Light itself. Motes of Dark similarly appear as black tetrahedrons, further suggesting a relationship between the Darkness and the titular Pyramids.
A single Pyramid resembling those that attacked the Sol System during the Collapse was discovered beneath the surface of the Moon in the years following the Red War. Even in its seemingly derelict state, this vessel exhibited profound Darkness-related abilities, such as the power to manifest Nightmares.
Recent investigations of the pyramid located on the Moon, as well as a newly-discovered region of the Black Garden, have revealed another motif that appears to be representative of, or at least associated with, the Darkness: statues of humanoid, veiled figures. Both statues thus far encountered are identical with seemingly feminine features, and are posed with faces turned upward and arms slightly outspread. The significance of these statues is currently unknown.
According to the Lore Book "Unveiling," which is seemingly narrated by the Darkness itself, both the Light and the Darkness have existed since before the beginning of time, and thus before the universe came into existence. Being unbound by the laws of causality, neither the Light or Darkness can be said to have a beginning. Rather, they are emergent properties of even more fundamental "mathematical structures" that in turn underlay reality itself. 
In the allegorical story recounted in the Lore Book, the Darkness and Light are referred to as the "Winnower" and "Gardener" respectively. The Winnower and Gardener occupied themselves by engaging one another in a game, analogous to a vastly more complex version of the "Game of Life" devised by Earth mathematician John Conway. This game would consistently arrive at an end-state dominated by a single, self-perpetuating pattern, which subsumed all others in the game. The winnower found this to be a pleasing outcome, but the gardener felt the pattern to be boring, and desired that the game endlessly produce novel patterns instead. In order to promote this novelty, the gardener transformed itself into a new "rule" within the game; the winnower did the same, to counteract the gardener’s efforts. These new rules were "set aside" from the other rules of the game, so that they could not be limited or influenced by them, but could manipulate the other rules to bring about their intended outcomes.
While the story above is laden with metaphor and open to interpretation, it appears to suggest that the universe is the "game" which the Light and Darkness played in the garden, that the rules of the game are the laws of physics as they are commonly understood, and that the "new rules" that the Light and Darkness became are the basis of paracausality.
The Distant Past
The earliest recorded mention of the Darkness known to the Guardians comes from the Books of Sorrow, a collection of records discovered aboard the Dreadnaught of Oryx, the Taken King. The books describe the origin of the Hive on the gas giant Fundament, where three daughters of a deposed king, Aurash, Xi Ro, and Sathona, set out to save their world from a prophesied apocalypse, and in their quest encountered the massive being known as the Leviathan. The Leviathan warned them against the temptation of "The Deep," a philosophy which held that "existence is the struggle to exist," and which it claimed would lead them only to ruin and calamity. It instead told them to pursue the philosophy of "The Sky," which encouraged collaboration to build a kinder, more prosperous world, even if it meant that some deaths and misfortunes must be allowed to occur. The sisters rejected the Leviathan's advice, and dove deep into the core of their world to find the five Worms: Eir, the Keeper of Order, Ur, the Ever-Hunger, Yul, the Honest Worm, Xol, Will of the Thousands, and Akka, the Worm of Secrets.
The Worm Gods promised to lend the sisters their paracausal powers, which they drew from the Deep, in return for the sisters becoming hosts for the Worms' larvae. The sisters agreed, and became the founders and God-rulers of what would one day become known as the Hive. The Worms taught the nascent Hive Gods of the Sword Logic, which they used to become ever more powerful.
Much later, the Hive God Auryx (who was once Aurash) slew the Worm God Akka, in order to obtain enough power from the Sword Logic to Take. He became thereafter known as Oryx, the Taken King. Some time later, he sought to commune with the Deep directly and prepared an Ogre as a sacrifice, which the Deep inhabited and used to converse with Oryx.
At an unknown time in the past, the Eliksni homeworld was visited by the Traveler, who ushered in an era of great prosperity and technological advancement. However, as with so many others before them, the Eliksni's Golden Age was ended when the Darkness found their civilization and decimated it, sending the Traveler fleeing and reducing the Eliskni to a race of interstellar nomads. This cataclysmic event became known in Eliksni history as the Whirlwind.
Meeting with Calus
During his exile aboard the Leviathan, the deposed Cabal Emperor Calus and his Loyalists encountered an anomalous void in deep space, described as a a total absence of "light, dark, life, death (...) anything, even of absence itself."  Donning a pressure gel suit, Calus went out to the exterior of the Leviathan to inspect his surroundings. He received a vision from the void, showing a fleet of unknown ships and a coming cataclysm that would annihilate all life in the universe. The void then told Calus that he would be its herald, to spread the news of the coming end.
Prior to the Collapse, a Human mining expedition on Luna was commissioned by Clovis Bray to discover the source of a mysterious wavelength buried beneath the Moon's crust. They discovered nearly twelve hundred meters deep a black sphere (identical to the one found centuries later), emitting neurochemical cascades, causes intrusive thoughts, insomnia, narcolepsy, nightmares, and auditory and visual hallucinations. The sphere was studied due to an outbound signal that was believed to be connected to a second Traveler, or even the origins of the Traveler. However, the expedition team, despite their best efforts, were eventually driven to madness and perished.
The only solid information on the Darkness gathered by humanity itself comes from an ISR report from centuries ago. At that time, the Darkness appeared just beyond the edge of the Solar System in what was termed as a "TRANSIENT NEAR EXTRASOLAR EVENT", with the only means of determining that a transient event had even occurred was through the detection of a 0.3 second displacement of gravity waves, as well as anomalous interactions in the local Higgs field traced to sterile neutrino scattering. An AI observing the event, the Warmind Rasputin, was able to derive some information, based solely on the phenomena that it could detect, but was unable to ascertain its mechanism. Rasputin conducted an "Omnibus analysis" of the event's effects on local spacetime, measuring a wide range of data points including direction, distance, range, speed and size as well as a comprehensive pattern analysis of the event itself. This last set of data showed, to a certainty of "9 Sigma", a definite and measurable pattern which had a complex, almost linguistic, structure that demonstrated both intelligence and intent.
In an attempt to better understand the phenomenon by recreating all of its known variables (also known as a "bootstrap simulation"), Rasputin reran the data through a series of iterations which included a number of randomized inputs, in a process known as a Monte Carlo analysis. Despite this, it was unable to derive any definitive information about the event or its source; however, it was able to determine that whatever was behind the event was clearly a complex structure, acting with clear purpose. Rasputin concluded that the source of the event not only failed to conform to any known phenomena in the Standard Model of particle physics (thus labeling it "acausal"), it was also directed, acting with anger, and almost certain to reach the solar system in a short amount of time. As a result, the previously issued Skyshock alert was upgraded to OCP:EXTINCTION, and a number of defensive protocols were added. The Darkness nonetheless swept through the Solar System, devastating human civilization in an event called the Collapse. Only due to the Traveler's intervention did humanity survive. Seven centuries later, the Darkness was set to return.
Events of Destiny
Centuries after the Collapse, a lone Guardian encountered the Exo Stranger on Venus, and was subsequently sent on a quest to find the Black Garden. After battling through the Sol Divisive Vex and reaching the center of the Garden, the Guardian discovered an entity, the Black Heart, that the Vex seemed to revere as an object of worship. The entity could not attack the Guardian directly, but attempted to defend itself by taking control of the Sol Progeny, three enormous Vex Minotaurs. After a protracted battle, the Black Heart was apparently destroyed along with its three Sol Progeny vessels. Upon the defeat of the Black Heart, the Black Garden was relocated to Mars from its prior location, and Light began to return to the dormant and damaged Traveler.
Following the battle in the Black Garden, Ikora Rey hypothesized that the Black Heart had been an aspect of the Darkness itself.
Events of Destiny 2
After the death of Dominus Ghaul and the restoration of the Light, a fleet of pyramid-shaped ships activated in response and started moving en masse toward the Milky Way. The shape of these ships bore a resemblance to the black tetrahedrons featured in the vision the Guardian experienced upon losing their Light, as well as the triangular shadow used to represent the Darkness in the opening cinematic.
Curse of Osiris
Following the reawakening of the Grasp of Nokris on Mars, the Guardian encountered the Worm God Xol, Will of the Thousands in the Penumbral Depths beneath Hellas Basin. There, the Worm God addressed the Guardian telepathically, claiming that the Guardian would "drown in the Deep".
The Scorn, a faction of undead Fallen first encountered by the Guardians in a mass breakout from the Prison of Elders, were created through the use of Darkness-tainted Ether by Fikrul, the Fanatic. This "Dark Ether" was first generated when Riven drew upon Uldren Sov's internal Darkness and infused it into Ether, bringing Fikrul back from death.
Eris Morn, after going missing prior to the Red War, stumbles upon a derelict alien vessel in the shape of a pyramid beneath the surface of the Moon. She discovered that the Darkness originating from the Pyramid was triggered into unleashing phantasmal Nightmares, and that the Hidden Swarm are harnessing them in a crimson citadel built above the ancient structure, near the Hellmouth, dubbed the Scarlet Keep. With Eris Morn as their guide and having the Vanguards approval on handling the situation, the Guardians made efforts in combating the Hive and the Darkness-powered Nightmares. Using the Hive's knowledge and the essence of the most powerful of Nightmares, Eris was able to forge armor that was capable of resisting the Nightmares.
Now prepared and with the Nightmares pushed back, the Guardian descends beneath the Scarlet Keep to the Pyramid. There, the Darkness itself manages to commandeer the Guardian's Ghost and opens the Pyramid, drawing in the Guardian. Inside, the Guardian faces off against some their most dangerous of foes: Ghaul, Crota and Fikrul. As they combat against the powerful Nightmares, the Darkness taunts the Guardians, through their Ghost, on the Light's failures and weaknesses. Despite this, the Guardian manages to overcome the Nightmares and reaches the center of the Pyramid.
The Guardian is presented with an Unknown Artifact and is subjected to a vision from within the Black Garden, where they encounter a doppelganger of themselves, apparently representing the Darkness itself. The Darkness addresses the Guardian, claiming that they are neither the Guardian's friend nor enemy, but instead their "salvation".
The Guardian brings the artifact to Eris and reports what they encountered in the Pyramid to the Vanguard. Utterly suspicious of what the Darkness speaks as their salvation, they take it as a confirmation that the Darkness is preparing to return. At the same time, Eris identifies a signal emitting from the artifact that leads deep within the Black Garden, to which a fireteam of Guardians was sent to investigate. What they found were remnants of the Pyramids that the Vex were attempting to harness with Vex Minds, which the fireteam destroyed and triggering the artifact to unlock. Eris began receiving messages directly from the Darkness to which she dismissed as attempts of manipulation, but nonetheless provided potentially valuable information into the origin of the Light, the Darkness, the Vex, and the Universe itself.
When Osiris confronted Rasputin regarding his allegiance to the Light or the Darkness, the Warmind directed him to an anomalous signal located in the Kuiper Belt. Upon arriving at the coordinates, Sagira reported feeling strange (in a manner similar to the Guardian's Ghost when approaching the Pyramid). Osiris then entered "an anomalous Maw" and returned with a "cold metal seed" before setting his ship on a course to return to the inner Solar System.
The Nature of the Darkness
Because so few survived the Collapse, the exact nature of the Darkness is unclear. Several theories as to what it is have been proposed by various scholars among the Guardians and other people of the City.
Toland, the Shattered understands the Darkness as the natural evolutionary endpoint of the entire universe. Since the beginning of time, everything has been in competition with everything else for no reason, other than that is the way of the universe: the law of the jungle. As atoms came into being, they defeated the "primordial broth" that preceded it. From atoms came stars, from stars came galaxies, worlds were created, and finally life arose, all due to the eternal competition.
Toland envisions a future where three archetypal "nations" compete with other: the first represents the perfect rule of law, the second represents knowledge made paramount, and the third represents sheer, total conquest. The third nation will succeed, inevitably, and in time will lead to a greater conflict between other nations which all represent the desire to conquer and dominate. The universe thus becomes a place where only the most ruthless entities succeed, to the point where they actively hunt and exterminate other life long before it can ever be a true threat. Finally, at the end of time there will be only one entity that rules the universe so absolutely that nothing will ever exist except by its consent, so it will rule forever. In essence, Toland asserts that this incontestable supremacy is the goal or driving force of the Darkness.
An alternative way of life, where law and knowledge and might worked together rather than competed, as in the case of humanity and the City, is little more than "the dream of small minds", and is unlikely to succeed against an entity of pure conquest.
Dreams of Alpha Lupi
The Dreams of Alpha Lupi poem contains a verse related to the Darkness, and possibly representative of the Darkness's own perspective. The verse explains that the universe, despite its vastness, is beholden to simple, enduring laws that shape it and give one the impression that it is inexhaustible and eternal. Life is an affront to the universe because it refuses to accept the limitations imposed on it by the natural order.
Ghost Fragment: Darkness 4 contains an excerpt of unknown origin, addressing an unknown recipient.
Its opening line: "This war is all there is for you." is also a line of dialog, spoken by Future War Cult representative Lakshmi-2, in which she asks the player what they would do if, "on some dreadful night, the Darkness were to come creeping up" and whisper the aforementioned statement to them. This would suggest that the excerpt in Ghost Fragment: Darkness 4 is the full text of that opening statement, indicating that the Darkness is addressing a Guardian, attempting to entice them into turning away from the Light, or at least plant the seeds of doubt. 
The Books of Sorrow
Following the arrival of Oryx, the Taken King in the Sol System, records were discovered aboard Oryx's Dreadnaught that provide new insight into the Darkness' history and nature. These records suggest that the Darkness is a manifestation or embodiment of the universal struggle for existence: whereas the Traveler seeks to nurture civilization by encouraging peace and mutual cooperation, the Darkness is dedicated to the annihilation of all beings and civilizations who do not possess the strength to survive in the face of conquest. Its eventual goal is to bring about a universe inhabited solely by those entities who have survived by defeating all opposition. To this end, it has destroyed countless planets and civilizations, either through direct action or through its disciples, such as the Hive. 
On one occasion, Oryx went deep into the Ascendant realm using the Tablets of Ruin and prepared an unborn Ogre, which he used as a vessel to summon and communicate with the Darkness, saying "I can see you in the sky. You are the waves, which are battles, and the battles are the waves. Come into this vessel I have prepared for you." It is here that the Darkness explained to Oryx how it viewed existence and how the universe functioned.
Aside from its philosophical/metaphysical characteristics, the Darkness and associated entities such as the Taken and the Dreadnaught are known to emit conventionally detectable particles and phenomena such as sterile neutrinos, axions, phaetons, and gravity waves, all of which are hypothetical particles or phenomena associated with dark matter. However, no direct connection between dark matter and the Darkness has been drawn.
The Darkness desires to sway the Guardians to its philosophy, as it acknowledges them as beings powerful enough to potentially become the "final shape" it wishes to bring about.
There are a handful of Guardians who fell into the darkness, notably Dredgen Yor. Yor came to desire taking innocent lives and sowing despair, claiming he had developed a hunger for it. Toland, the Shattered claimed to be able to communicate with the Darkness, whereas Eris Morn was able to survive and escape the Hellmouth wielding the very power she fought against. Eris had one time even used the power of Darkness to rescue the Guardian from being trapped in the Oversoul Throne. As noted above, it would appear that the Darkness attempts to turn Guardians away from the Light. One example is when Pujari had his vision of the Black Garden; the vision made him doubt the righteousness of the Guardians when the vision told him, "You are a dead thing made by a dead power in the shape of the dead. All you will ever do is kill. You do not belong here. This is a place of life." This hearkens back to Ghost Fragment: Darkness 4, where the Darkness calls Guardians "a dead husk" animated by "an ancient corpse," referring to the "dead" Traveler.
According to Fallen Lore, their civilization was destroyed when the Great Machine left them and the Whirlwind tore the sky. The Great Machine and Whirlwind are the Traveler and Darkness respectively, making the Fallen one of the races to survive the Darkness's devastation, and confirming Saint-14's hypothesis as partially true. A reclaimed (though corrupted) Fallen story mentions that the Fallen respect what they cannot steal from, which is "why the dark is worthy of love beyond all other love that astonishing ability to evade being robbed". When The Consul refers to the Traveler as "this great machine", this provides confirmation that the Fallen were indeed referencing the Traveler, as the Consul notes "I believe other civilizations would be more precise in their naming", perhaps a deliberate reference to the Fallen.
The Hive clearly derive their power from the Darkness, and feed on and hate the Light. The Hive religion, based around the 'Sword-Logic', closely matches Toland's theory above. The Hive made a pact with their highest gods, the Worms, who grant the powers of Darkness to the Hive. They worship the Darkness as a force of destruction, and their worship has taken the form of chronicling every planet they have seen the Darkness destroy, a massive archive known as "the World's Grave" found deep within the Hellmouth. The Hive religion has a pantheon of other gods, but one in particular, Oryx, is of significant importance. Oryx commands both the Hive and his own personal army, the Taken, by using the power of the Darkness (see below). This power he stole from his patron deity, the worm Akka, so that he could access the Darkness directly like the Worms could. Osiris prophesied that should the Spawn of Crota succeed in snuffing out the worlds of Light, Oryx's coming would be unfettered, reminiscent of the Darkness's predicted arrival.
The majority of Vex avoid the Darkness, due to a combination of their inability to compute anything paracausal and the fact that their ultimate goal of Convergence (an end state of the universe where only the Vex exist) means they would have to overcome the Darkness when it is utterly impossible. The Vex of the Sol Divisive however, worship the Black Heart, an aspect of the Darkness. The Monist's theory about the Darkness matches the Vex's goal of programming themselves into the fabric of the universe. Osiris theorized that some Vex technology is tuned to react to the presence of the Darkness.
According to Osiris, the Vex's goal of Convergence would result in a universe where neither Darkness nor Light existed any longer. This suggests that the Vex do not revere the Darkness as a whole, or that they view the worship of the Darkness as a means to an end of their goals.
Through unknown means, the Sol Inherent, a powerful subdivision of the Sol Divisive, appear to have been exposed to the Darkness and changed by it somehow, causing them to take on malformed, organic, visceral appearances. Upon the deactivation of the Sanctified Mind, Sol Inherent, a structure behind it opens up to reveal a statue identical to the one found on the Pyramid, and is itself hidden by a geometrical platform not unlike the architecture of the Pyramids themselves. The significance of this is unknown.
According to a dead Ghost, the Cabal are secretly running away from a terrible power following them, presumably the Darkness.
The Cabal had some texts on their athenaeum worlds that theorized about the Darkness, with the little concrete information the texts contained about the Darkness having been recorded from the "dreams of worms". 
Evocate-General Umun'arath, Primus of All Legions, once said in reference to Cabal civilization and culture that "The war is all there is. All this is just logistical support." This echoes the above-mentioned phrase used by the Darkness and the Future War Cult.
During his exile, the former emperor Calus sought the means to defeat his usurper Ghaul by exploring the secret places of the galaxy and pushing the limits of reason once he regains control of the Leviathan. He eventually crossed paths with a perfect void at the edge of the universe, most likely the Darkness itself, and resolved to become its herald. He would watch it consume the light of the universe, and all among the last of the living would know his greatness. Calus also knows of the pact that Oryx made with the Worms, comparing it to Ghaul's takeover of the legions. Calus' strange powers and his claims of "knowing the truth" about the galaxy and his mockery of the Guardians' "feeble light" implies that he may have some knowledge of the inner workings of the Darkness.
Following the disruption of time on Mercury, several Red Legion Cabal were observed manifesting what appeared to be Darkness-derived powers. It is unclear how they acquired these abilities.
Awoken & Jovians
The Awoken were once humans, but were transformed during the Collapse upon reaching the edge of the asteroid belt. Beyond the belt, "the edge of the Deep Black", the system is totally immersed in Darkness. The events of their transformation were further detailed in the Lore book "Marasenna", which reveals that the Awoken were captured in a kugelblitz formed by the Darkness and Light battling one another in the vicinity of the Yang Liwei. As a result of this event, the Awoken were reborn as beings containing both Light and Darkness within themselves.
Likewise, the Jovians, particularly people such as Xûr, were once humans but were more heavily transformed than the Awoken by the Collapse. Legends do not offer a concrete explanation but imply that they were shaped by the Darkness.; its safe to assume the same happened to the Awoken, but to a lesser extent, due to their proximity. Though they are willing to aid humanity through Xûr, the Light is a foreign concept to Jovians and causes them pain.
Through Oryx, the Darkness from the Ascendant Plane can take direct possession of other races in the form of an ontopathogenic virus, thus creating the Taken. When a being is Taken, they are warped into the Ascendant Plane, where they are paracausally transformed by the Darkness into spectral ghost-like entities of their former selves. They are convinced that being Taken has "liberated" them from their fears and grants them new powers. The ability to create Taken was derived from the Worm God Akka, but Oryx killed him and stole as much knowledge as he could. Reports from the Ecumene fighting Taken described them of having abilities that are "physically illegal," meaning their very existence broke all known laws of physics.
Nightmares are apparitions created by the Darkness, usually manifesting to a given individual as a former foe or comrade whose memory is associated with fear, regret or other emotional trauma.
List of appearances