To prettify symbols in c-mode using
prettify-symbols-alist, you can add a hook in your Emacs
configuration file (e.g., init.el or .emacs).
Here's an example:
This hook will enable prettify-symbols-mode in
c-mode and replace common C operators with prettier
symbols. Adjust the prettify-symbols-alist as needed for
your preferences.
The LLM model is to big to put into the device, if there any batter
architecture to make the model small enough to put to the device and
keep the precision and inference performance.
The LLM model embedding the word to very high dimension, in order to
do better classification. But it will affected by the curse
of dimensionality. So we need deep neural net work(more parameters)
and more training data to avoid overfitting.
But each time put some words into the space, may be adding one new
dimension only useful for related words, and for most other words will
add one no useful dimension. And the whole vector space is sparse and
need more training data and complex model(more parameters).
We can use the idea of mixture of experts, they form one graph, and
some experts are independent, but some has relations between each other,
can be connected by edges. So the problem becomes make each experts'
dimension as small as possible, and the duplication features in
different experts as small as possible. We convert one high dimension
sparse LLM to many row dimension dense experts. Just like the symbol
table design in compiler, we can use one big flatten symbol table, but
the scope level and scope name in each scope
are same and can be eliminated, so traditional implementation of the
symbol table use the chain of scopes hash table, make a more compact
memory.
Design
A graph to represent the
model, is all
experts, and is the edge
of the two experts.
Partitioning
Take the compute system as the whole world. We can split experts
to
operating system
database
compiler
programming language
distributed
network
micro architecture
application
electronic
common knowledge
Once a word file system coming, will be both added to
operating system and databse, and if a word
computer arrived, it will be put to the
common knowledge. And all experts depend on the
common knowledge.
Evolve
All the experts can be version controlled, and can evolve,
refactoring. Like add more words, add dimension, remove words to other
experts, extract more experts. Just like the package manage system, all
experts are packages.
Query(inference)
The query convert to tokens, and send to the related experts,
exploit method use dependent experts to combine the
results, exploare method to query other experts to find
results too. The results contian all the experts, the sub graph forms a
path. The path can be cached.
Online learning
Support real time input sentences. The tokenizer first to
classification, and put to different experts, batch will trigger insert.
Insert data to the experts, and update the path.
During the period 1904 to1947, the vacuum tube was the electronic
device of interest and development. In 1904, the vacuum-tube diode was
introduced by J. A. Fleming. Shortly thereafter, in 1906, Lee De Forest
added a third element, called the control grid, to the vacuum
diode, resulting in the first amplifier, the triode. In the
following years, radio and television provided great stimulation to the
tube industry. Production rose from about 1 million tubes in 1922 to
about 100 million in 1937. In the early 1930s the four- element tetrode
and the five-element pentode gained prominence in the electron-tube
industry. In the years to follow, the industry became one of primary
importance, and rapid advances were made in design, manufacturing
techniques, high-power and high-frequency applications, and
miniaturization.
三极管的发明，和工业界的需求有密切联系。radio and television provided great stimulation to the tube industry.
看现在的AI需求，对于新计算机体系结构也会有新的需求，比如memory
system的演进，可能会有比较革命性的变革，比如pim。
On December 23, 1947, however, the electronics industry was to
experience the advent of a completely new direction of interest and
development. It was on the afternoon of this day that Dr. S. William
Shockley, Walter H. Brattain, and John Bardeen demonstrated the
amplifying action of the first transistor at the Bell Telephone
Laboratories. The original transistor (a point-contact transistor) is
shown in Fig. 2. The advantages of this three-terminal solid-state
device over the tube were immediately obvious: It was smaller and
lightweight; it had no heater requirement or heater loss; it had a
rugged construction; it was more efficient since less power was absorbed
by the device itself; it was instantly available for use, requiring no
warm-up period; and lower operating voltages were possible. Note that
this chapter is our first discussion of devices with three or more
terminals. You will find that all amplifiers (devices that increase the
voltage, current, or power level) have at least three terminals, with
one controlling the flow or potential between the other two.
The circuit consists of a resistor in series with the voltage source
connected to the inverting
input of the OpAmp. The non-inverting input of the OpAmp is short
circuited to ground(common). A resistor is connected to the output and
provides a negative feedback path to the inverting input terminal.
and Since is connected to
the common or ground terminal, Node 1 is said to be a virtual ground due to the virtual short
circuit between the inverting and non-inverting terminals (which is
grounded) as defined by the voltage constraints, The node voltage method of analysis is applied at node 1,
By appling Equation , obtained from the virtual
short circuit, and the constraint on the current as defined in Equation, Equation is simplified to Solving for the voltage gain, , Notice that the voltage gain is dependent only on the ratio of
the resistors external to the OpAmp, and . The amplifier increases the
amplitude of the input signal by this ratio. The negative sign in the
voltage gain indicates an inversion in the signal.
The output voltage is also constrained by the supply voltages and , Using Equation , the maximum resistor ratio
for a given input voltage
is
实验
根据下图，计算增益和。如果，输出是多少？多大的输入电压幅值，会造成amplifier饱和？
inverting opamp example
Solution:
The output voltage is
independent of the load resistor, , because of the low output resistance
of the OpAmp. Therefore, the gain of amplifier is Using KCL at node a, The currents and
are and Solving for yields,
For and input voltage signal of 由于我使用的是模块，具体参考双电源供电。
