$0.00

No products in the cart.

sales@glitchdata.com

$0.00

No products in the cart.

Home Blog

The Sanctity of Access

0

The collective store of knowledge has been generated by human in forums, reddit, stackoverflow, websites, Wiki, and search engines. We have used this to train ChatGPT. Since GPT4, the generation of knowledge is moving from the public-sphere into a direct private chat. This is a problem.

So the generation and capture of knowledge is now privately sourced and pooled into a Machine by-passing the human public domain.

Where will the next AI model get its training data from? GPT-Next will be trained on legacy data.

This raises multiple questions related to Ethics, the sanctity of Data Access, and an increasing importance to legislate for public data.

AI will become a dominant source of knowledge simply by virtue-of-growth. It depends on training data which could become unavailable, or monopolised, or licensed by a chatbot.

Trusting the output from AI is just as alarming. If we lose access to the training data, and propagate AI outputs we will lose the collective memory of human-generated thinking and lose the ability to validate AI. Just like search, humanity will be told what to think, how to think, and programmed by AI.

So to avoid a dangerous feedback loop, we need to individually, and collectively support (and extend) the public domain.

What is ChatGPT?

0

So 2023 started with a big buzz on ChatGPT. OpenAI announced ChatGPT availability just before Christmas, and soon the internet was abuzz with excitement. ChatGPT app gained some 1 million users after 5 days, and 10million users after 40 days. This is way faster that previous upstarts like Instagram, Twitter etc…

So what exactly is ChatGPT? It’s an intelligent chatbot driven by the GPT-3 model. GPT stand for “Generative Pre-training Transformer”. Essentially, its a weighted Neural Network with 175 billion weighted connections. This is the largest trained neural network to-date. Microsoft maintains the next largest “Turing-NLG” model at 17 billion connections. GPT-4 will have ~500 billion connections. The human brain has approximately 86 billion neurons (sometimes less). The model includes reinforcement learning, and supervised learning. There are also aspects to the architecture which include components like The Encoder, The Decoder, Language Model, Pre-trainers, Fine-tuners etc…

So ChatGPT-3 has sparked lots of interesting conversations and use cases. People have started to apply it to work, assignments, content-generation, coding, and general life questions. This was made possible through the training of the model with a terabyte of data. Will it surpass the Google search engine simply by being able to create a more intelligent answer beyond the search result?

Architecture

  1. The Transformer architecture: The Transformer architecture is the foundation of ChatGPT. It is a neural network architecture that uses self-attention mechanisms to process input sequences. The transformer architecture is able to handle input sequences of varying lengths and allows for parallel processing of the input.
  2. The Encoder: The Encoder is composed of multiple layers of self-attention and feed-forward neural networks. It processes and understands the input text.
  3. The Decoder: The decoder is also composed of multiple layers of self-attention and feed-forward neural networks. It generates the output text.
  4. The Language Model Head: The language model head is a linear layer with weights that are learned during pre-training. It is used to predict the next token in the sequence, given the previous tokens.
  5. The Dialogue Generation Head: The dialogue generation head is a linear layer with weights that are learned during fine-tuning the model on conversational data. It is used to generate the response to a given prompt in the context of a dialogue.
  6. Pre-training: ChatGPT is pre-trained on a large dataset of text, which enables it to generate human-like text in response to a given prompt.
  7. Fine-Tuning: The model is fine-tuned on conversational data to improve its ability to generate responses in the context of a dialogue.

Training

Training is another interesting consideration. How much data is needed to train GPT-3? (570 gigabytes of text). How long did it take? ChatGPT is trained (censored/biased) to avoid returning harmful answers. As the size of the model increases, training time and data will also significantly increase.

Ethics

A concern for the future is that ChatGPT model are biased. Another concern is that humans continue to get progressively obsolete. Ethics for AI is going to be crucial for humanity.

ChatGPT Trajectory

  1. GPT-1 (Generative Pre-trained Transformer 1) was the first version (in June 2018) of ChatGPT released by OpenAI. It was pre-trained on a dataset of 40GB of text data and had a capacity of 1.5 billion parameters.
  2. GPT-2 (Generative Pre-trained Transformer 2) was released shortly after in Feb 2019. It was pre-trained on a much larger dataset of 570GB of text data and had a capacity of 1.5 trillion parameters, making it ten times larger than GPT-1.
  3. GPT-3 (Generative Pre-trained Transformer 3) was released in 2020. It was pre-trained on a massive dataset of 570GB of text data and had a capacity of 175 billion parameters. It was fine-tuned for a wide range of language tasks, such as text generation, language translation, and question answering.
  4. GPT-4 (Generative Pre-trained Transformer 4) was released in 2021, it was pre-trained on a massive dataset of many terabytes of text data and had a capacity of over 500 billion parameters. It was fine-tuned for a wide range of language tasks, such as text generation, language translation, and question answering with even more accuracy and fluency than GPT-3.

Current Limitations of ChatGPT

  • GPT-3 lacks long-term memory — the model does not learn anything from long-term interactions like humans.
  • Lacks interpretability — this is a problem that affects extremely large and complex in general. GPT-3 is so large that it is difficult to interpret or explain the output that it produces.
  • Limited input size — transformers have a fixed maximum input size and this means that prompts that GPT-3 can deal with cannot be longer than a few sentences.
  • Slow inference time — because GPT-3 is so large, it takes more time for the model to produce predictions. Imagine how long GPT-4 will take?
  • GPT-3 suffers from bias — all models are only as good as the data that was used to train them and GPT-3 is no exception. The data for GPT-3 and other large language models contain biases. This already intentionally includes “hate”-speech biases, religious biases, political biases.
  • Training Time – With GPT-4 coming-in with 500B parameters, we can see a 2.8x increase in parameters. Is the trajectory slowing down?

Here are some links to articles:

The Battlefield

0

World Economic Forum 2023

0

World Economic Forum 2023, Davos: The World Economic Forum (WEF) meet is scheduled to be held in Davos, Switzerland, from January 16 to January 20, 2023. The forum has returned to its traditional January slot under the theme of ‘Cooperation in a Fragmented World’, where leaders from industry, and government fly in on their private jets.

The WEF aims to look at how they can tackle the imponderable and interlinked challenges the world is facing and find solutions through public-private cooperation. However, it’s important for countries to ensure that decisions by governments are not affected by foreign influence and are made by elected-officials representing the populace.

The event will be hosted by the World Economic Forum in cooperation with the Swiss government and the Canton of Graubünden.

Items on the agenda are:

  • Central Bank Digital Currency
  • Dis-information
  • De-globalisation (oxymoron)
  • Biotech for food security
  • Cybersecurity
  • Data Privacy

Independent Media currently on-site:

  • True North
  • Rebel Media

Commentary:

FTX – Sam Bankman-Fried (SBF) arrested

0

Police in the Bahamas have arrested the founder of collapsed cryptocurrency exchange FTX after criminal charges were filed in the US.

Sam Bankman-Fried was the lead scammer who laundered money through FTX. The company that was valued at $47B before speculation FTX was running insolvent led to a liquidity crunch that ultimately brought down the exchange and has triggered a contagion effect throughout the world of cryptocurrency.

In a statement, Bahamian Attorney General Ryan Pinder said the country is expecting an extradition request to follow shortly.

“As a result of the notification received and the material provided therewith, it was deemed appropriate for the Attorney General to seek SBF’s arrest and hold him in custody pursuant to our nation’s Extradition Act,” Pinder said.

“At such a time as a formal request for extradition is made, the Bahamas intends to process it promptly, pursuant to Bahamian law and its treaty obligations with the United States.”

Despite accusation that Bankman-Fried defrauded FTX’s customers, he has remained in the public eye making public appearances from the Bahamas where his now bankrupt company was headquartered. Sam desperately needs to remain in the public eye to avoid being “whacked”. Several crypto CEOs have recently been found dead.

Two weeks ago, he appeared at a New York Times conference, where he said he “did not try to commit fraud on anyone”.

Last week, SBF agreed to front to a US House Committee and tweeted that he thought of himself “as a model CEO, who wouldn’t become lazy or disconnected”.

“I’m sorry,” he said. “Hopefully people can learn from the difference between who I was and who I could have been.”

The sudden collapse of FTX impacted thousands of Australians who had funds on the exchange, and has created market conditions that recently led to one of the country’s largest exchanges Swyftx laying off 30% of staff.

Current CEO of FTX John Ray III – who is administering the company through its bankrupty – also oversaw the infamous Enron bankruptcy. FTX was a donated large sums of money to the Democrat party, and to certain Republican leaders (like Mitch McConnell).

“The FTX Group’s collapse appears to stem from the absolute concentration of control in the hands of a very small group of grossly inexperienced and unsophisticated individuals who failed to implement virtually any of the systems or controls that are necessary for a company that is entrusted with other people’s money or assets.”

This looks like a page out of the money laundering playbook.

Defence Cyber Strategy

0

Future conflict will involve sophisticated cyber warfare. Nations across the globe have recognised the strategic value and asymmetric advantage of investment in offensive cyber capabilities. They continue to evolve and advance their capabilities, contributing to an evolving strategic environment. In this context, cyber security is now one of our most critical tools to defend our people, capabilities, and ultimately, our nation.

On 31 August 2022, the Assistant Minister for Defence and Assistant Minister for Veterans’ Affairs, the Hon Matt Thistlethwaite released the Defence Cyber Security Strategy, outlining how Defence will strengthen its cyber security posture over the next ten years.

The Strategy details how Defence will combat cyber threats and ensure its capabilities are secure against attacks from adversaries. It presents the path to a cyber resilient Defence and the principles to maintain a strong cyber security posture in a shifting strategic environment.

Ultimately, the Strategy will contribute to a high-performing One Defence enterprise that can continue to deliver on its mission of defending Australia and its national interests.

The Strategy was released alongside the complementary 2022 Defence Information and Communications Technology Strategy.

Read more:

Patagonia evades Climate Change?

0

Aussie company Patagonia is donating its profits to fight climate change. You’ll see that splashed across the news.

However, could the real story be tax evasion?

You heard about Patagonia and their big donation? Interested in another perspective?

The Patagonia headline is “Billionaire Gives Away Company to Fight Climate Change”.

Wow that’s awesome right⁉️🔥

However, there’s more to it than most realize.

Yes 98% of the company is now owned by a non-profit that will leverage the $100M in profits to fight the environmental crisis.

BUT Yvon Chouinard and family save $700 MILLION in taxes.

He moved all his family voting stock, appx 2% of its total shares to an entity called Patagonia Purpose Trust. This entity now can make unlimited political donations in the U.S.

The deal is designed for the ultra-wealthy to use non-profits to exert political influence long past their lifetime.

He won’t have to pay $700M in capital gains taxes for selling a $3 Billion company.

He avoids paying a 40% estate and gift tax when transferring large fortunes to heirs.

When asked for comment on the tax implications… crickets. 🦗😶

I’ll share the Bloomberg article here and in the comments.

 https://lnkd.in/gdBDRJBa

I love the idea of positively impacting the planet. But “battling climate change” can mean a lot of things.

What do you think about all this⁉️😲👇🏾

Insights for Good

0

The illusion of insights

Data Breaches 2022 in Australia

0

University of Western Australia – August 2022

Uber – July 2022

Perth Festival, Black Swan State Theatre Company – July 2022

Victorian Government – July 2022

Woolworths – July 2022

Marriott – July 2022

Mangatoon – July 2022

China Police – July 2022

Deakin University – July 2022

AMD – July 2022

OpenSea – July 2022

iCare – June 2022

Department of Home Affairs – May 2022

NDIS – May 2022

Spirit Super – May 2022

APAC – May 2022

Facebook – May 2022

South Australian Government – May 2022

National Tertiary Education Union – May 2022

Transport for NSW – May 2022

Coca-Cola – April 2022

Top Data Breaches and Cyber Attacks of 2022

Panasonic – April 2022

Block (ASX:SQ2) – April 2022

Warrnambool Council – March 2022

OKTA – March 2022

Microsoft – March 2022

Ubisoft – March 2022

Nvidia – March 2022

Samsung – March 2022

Australian Data Breaches Report

Toyota Motor – March 2022

OAIC Report – February 2022

NSW Government – February 2022

News Corp – February 2022

CFMMEUR – February 2022

Red Cross Australia – January 2022

TfNSW (Accellion) – January 2022

FlexBooker – January 2022

Bunnings – January 2022

MITRE ATT&CK Framework

0

MITRE ATT&CK® stands for MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK). The MITRE ATT&CK framework is a curated knowledge base and model for cyber adversary behavior, reflecting the various phases of an adversary’s attack lifecycle and the platforms they are known to target. The tactics and techniques abstraction in the model provide a common taxonomy of individual adversary actions understood by both offensive and defensive sides of cybersecurity. It also provides an appropriate level of categorization for adversary action and specific ways of defending against it.

The behavioral model presented by ATT&CK contains the following core components:

  • Tactics denoting short-term, tactical adversary goals during an attack (the columns);
  • Techniques describing the means by which adversaries achieve tactical goals (the individual cells); and
  • Documented adversary usage of techniques and other metadata (linked to techniques).

MITRE ATT&CK was created in 2013 as a result of MITRE’s Fort Meade Experiment (FMX) where researchers emulated both adversary and defender behavior in an effort to improve post-compromise detection of threats through telemetry sensing and behavioral analysis. The key question for the researchers was “How well are we doing at detecting documented adversary behavior?” To answer that question, the researchers developed ATT&CK, which was used as a tool to categorize adversary behavior.

What is in the MITRE ATT&CK Matrix?

The MITRE ATT&CK matrix contains a set of techniques used by adversaries to accomplish a specific objective. Those objectives are categorized as tactics in the ATT&CK Matrix. The objectives are presented linearly from the point of reconnaissance to the final goal of exfiltration or “impact”. Looking at the broadest version of ATT&CK for Enterprise, which includes Windows, macOS, Linux, PRE, Azure AD, Office 365, Google Workspace, SaaS, IaaS, Network, and Containers, the following adversary tactics are categorized:

  1. Reconnaissance: gathering information to plan future adversary operations, i.e., information about the target organization
  2. Resource Development: establishing resources to support operations, i.e., setting up command and control infrastructure
  3. Initial Access: trying to get into your network, i.e., spear phishing
  4. Execution: trying the run malicious code, i.e., running a remote access tool
  5. Persistence: trying to maintain their foothold, i.e., changing configurations
  6. Privilege Escalation: trying to gain higher-level permissions, i.e., leveraging a vulnerability to elevate access
  7. Defense Evasion: trying to avoid being detected, i.e., using trusted processes to hide malware
  8. Credential Access: stealing accounts names and passwords, i.e., keylogging
  9. Discovery: trying to figure out your environment, i.e., exploring what they can control
  10. Lateral Movement: moving through your environment, i.e., using legitimate credentials to pivot through multiple systems
  11. Collection: gathering data of interest to the adversary goal, i.e., accessing data in cloud storage
  12. Command and Control: communicating with compromised systems to control them, i.e., mimicking normal web traffic to communicate with a victim network
  13. Exfiltration: stealing data, i.e., transfer data to cloud account
  14. Impact: manipulate, interrupt, or destroy systems and data, i.e., encrypting data with ransomware

Within each tactic of the MITRE ATT&CK matrix there are adversary techniques, which describe the actual activity carried out by the adversary. Some techniques have sub-techniques that explain how an adversary carries out a specific technique in greater detail. The full ATT&CK Matrix for Enterprise from the MITRE ATT&CK navigator is represented at https://attack.mitre.org

Read more