User login


You are here

Textbook on linear algebra

Zhigang Suo's picture

Linear algebra is significant to many aspects of mechanics.  For some years I have been using the book by Shilov.  But this book may or may not be a good one to recommend to a student, depending on his or her prior experience.  On StackExchange Mathematics, there are several excellent threads discussing textbooks of linear algebra.  A particular recommendation was made for textbooks of linear algebra at three levels.  Do you have any recommendations?


arash_yavari's picture

Dear Zhigang:

A classic on this subject is:

P. R. Halmos, Finite-Dimensional Vector Spaces, Springer, 1987.

I use parts of the following book in the applied math course I teach at Georgia Tech.

A. W. Naylor and G. R. Sell, Linear Operator Theory in Engineering and Science,
Springer, 2000.

Jim Knowles wrote a more applied book that may be more appropriate for beginners.


Zhigang Suo's picture

Dear Arash:  Many thanks.  I have the books by Halmos and Knowles, and like them both.  I also like the book by Gelfand, which Knowels recommended highly.  I did not know about the book by Naylor and Sell.  The table of contents of the book at Amazon look very interesting.  Do you and your students like the book?

Chapter IV of Gelfand is devoted to tensors.  He did not used components to define a tensor, but use a multilinear function, with the domain being the product of a vector space and its dual.  I like the general approach of using linear maps to define tensors, but am not sure multilinear function is general enough for our purpose.  For example, the stress tensor is a linear map that sends the area vector to the force acting on the area.  The domain and the range of this map are two distinct vector spaces.

What do you think? 

arash_yavari's picture

Dear Zhigang:

Yes, I like this book. I actually read it when I was a graduate student at Caltech.

I think that definition is fine and general enough even for stress. 1) If you consider Cauchy stress it is a tensor. It is a linear map that given a unit vector (normal to an area element) gives you the force acting on that area. Force is a convector, which can be identified with vectors using the trivial Euclidean metric (or even in more general spaces). 2) If you consider the first Piola-Kirchhoff stress it is a two-point tensor. It is a linear map that given a unit vector in the reference configuration (normal to an undeformed area element) gives you the force on the deformed area (a convector in the ambient space). This is a two-point tensor and obviously conjugate to the deformation gradient.

Again the "domain" and "range" need not be the same.


Zhigang Suo's picture

Dear Arash:  Sounds like that you and I think in the same way.  I have just finished teaching a course on solid mechanics.  My notes describe both the nominal stress and the true stress in the way as you outlined above.  Thus, the nominal stress is the linear map that sends the area vector in the reference state to the force acting on the area in the current state.  The domain and the range are two distinct vector spaces.

Pradeep Sharma's picture

Dear Zhigang, I have followed the linear algebra discussion on iMechanica with quite a bit of interest. Although I don't have anything meaningful to contribute beyond what has already been said, I would like to make a book recommendation.

When I joined the University of Houston as a young assistant professor, Lewis Wheeler used to teach our required graduate math course sequence (--which I later taught as well). He is a big believer in teaching linear algebra in depth and also successfully convinced me of its importance for mechanicians. He refers to several books in the math course however, when I had asked him for an elementary text, he recommended the following; which he used heavily: "Linear Algebra Done Right":

I was amused to see on your post that there is also a book with precisely the opposite title! I had read the Linear Algebra Done many moons ago and liked it quite a bit. If I recall correctly, it does not quite get into tensors but lays out all the foundational material both rigorously and an easy to understand manner

Zhigang Suo's picture

In a recent book, Jeevanjee wrote a short section (3.4) involving two distinct vector spaces, but in a very specifc way.  Here is a question.  Can we simply say that any linear map from one or more vector spaces to yet another vector space is a tensor?  That is, linear map and tensor are synonymous.

arash_yavari's picture

Dear Zhigang:

I haven't read this book but tensors are multilinear maps. You can certainly have more than one vector space. An example in continuum mechanics is "deformation gradient", which is a linear map from a vector space in the reference configuration (tangent space at a given point X in the reference configuration) to another vector space in the ambient space (tangent space at the image of X in the ambient space). This is an example of a two-point tensor.


Zhigang Suo's picture

Dear Arash:  Many thanks.  If tensors are the same as linear maps between vector spaces, we may as well eliminate the word "tensors", or just use the word "tensors" as shorthand for "linear maps between vector spaces".  Do you also think this way?  

arash_yavari's picture

Yes, this is correct.


Amit Acharya's picture

Zhigang - What you say above is correct except I would add that the spaces in question are finite dimensional. Bounded linear operators on infinite dimensional spaces come close to tensors on finite dimensional spaces, but even then it is a bit of a different ball-game.

I uploaded a couple of set of notes by Don Carlson  - I had to use optimized pdf as the file sizes were too large otherwise. If you take a look at Chapter A.4, the connection betweeen the multilinear approach and viewing tensors as a linear transformation between a f-d vector space and spaces of higher-order tensors on the same space are all spelled out.

If there is interest, I can upload the rest of the notes too. It is absolutely beautiful stuff for the beginner and has changed the course of at least one life....

Zhigang Suo's picture

Dear Amit:  Thank you for uploding Carlson's notes.  I have downloded them.

Zhigang Suo's picture

Dear Amit:  I went through the Carlson notes that you posted online.  Wonderful notes from a master!  Thank you so much for your kindness.  Would you please post the rest of the notes?  I'm eager to study them.

In Chapter A4, Carlson defined a tensor of order m on a linear space of dimension n.  He offered several definitions.  According to one definition, a tensor is a multilinear function that maps several elements in the linear space and its dual space to a scalar in a one-dimensional linear space.  

Incidentally, this definition was also adopted by Gelfand and by Jeevanjee.

I feel that this definition is too restrictive for mechanicians.  We routinely use linear maps that map elements in different linear spaces.  For example, stress is a linear map that maps an area vector to the force acting on the area.  The area vector is an element in a linear space--that is, a linear combination of two area vectors is yet another area vector.  The force is an element in another linear space.  The two linear spaces are distinct:  we do not form a linear combination of an area vector and a force.

As mentioned in the discussion with Arash, I feel that we should simply make the word tensor synonymous to the phrase "multilinear map that maps elements in several linear spaces to an element in yet another linear space".  The phrase is long, but the concept is a simple one.

This more general definition will force us to make a separate choice of basis for each linear space.  Of course, in some cases, we can make bases in different linear spaces coincide.  

This definition will also allow the different linear spaces involved in the mapping to have different dimensions.

Not sure if this more general definition is acceptable to people. 

Amit Acharya's picture

Zhigang: The definition of (higher-order) tensors that I find useful and have used extensively in mechanics is A 4.1.1. out of Carlson, not the multilinear function mapping to R. One feature of that use of tensors is that the domain and co-domain are not the same vector spaces....

What goes by the name of a linear transformation (e.g. Carlson A 2.4) has the generality you seek; in fact even more as no notion of dimension need be associated with domain and co-domain - and this is useful in mechanics too as with linear differential operators.Tensors use a little more structure, thus in my opinion a different name is OK.

Anyway, in the end, these are all definitions and the important thing is to understand them well enough to do something reasonable with them without too much fuss......

Zhigang Suo's picture

I quote from the Carlson notes:

"Vn is an n-dimensional inner-product space. 

Definition A4.1.1.  A tensor of roder m on Vn is a linear function on Vn that maps elements of Vn into tensors of order (m - 1).

We take tensor of order 0 to be scalar; i.e., elements of R, regarded as a linear space.  The tensors of order 1 are linear functions on Vn to R;  i.e., linear functionals."

I feel that this definition is too restrictive for mechanitians.  In this definition, a single linear space Vn occupies a special place.  For example, the definition does not cover the situation in which we have two distinct vector spaces U and V.  

In particular, the definition does not allow the stress to be a tensor. Using force balance, one can show that stress is a linear map that maps an area vector to the force acting on the area.  The area vector is an element in a vector space, and the force is an element in another vector space.  The two vector spaces are distinct:  We do not form a linear combination of an area vector and a force.

Amit Acharya's picture

Zhigang: May be I misunderstand the point you are trying to make and, then again, may be not. As you know, the tensors of most common usage in classical continuum mechanics is the tensor of order 2 that takes vectors from V_3 to V_3, this being the space in which we think the stress tensor belongs. In what you say above, do you want distinct vector spaces for domain and co-domain? Then, even the tensor of order 1 allows these to be different. If you are thinking of the domain to be a Cartesian product, that can also be handled as a fancy V_n.

Agreed that if one gets very picky,  one could say why and how is it valid to write (when one chooses to do so) position vectors, velocities, tractions etc. all in some common basis for (the translation space of) a 3-d Euclidean space by thinking of them as members of the same vector space; it is with this identification that the classical definition of stress as a second-order tensor makes sense. But as we all know, this works out just fine, and I admit I have not thought about this seriously. I believe that to do this precisely, one implicitly utilizes the isomorphism that necessarily exists between the 3-d vector spaces for each of these quantities. If I understand you correctly, you would prefer to think of say position vectors and velocities also as members of separate vector spaces?

Btw, I have uploaded the rest of Carlson's notes as you requested. Naturally, Balance of linear momentum with stress as a 2nd order tensor between V_3 to V_3 is worked out there in gory detail.

 Also, the few pages of introductory Continuum Mechanics in Hill's "Aspects of Invariance in Solid Mechanics" article are fantastic and complementary to say Carlson's treatment. In particular, it does a great job of explaining convected derivatives for people without background in differential geometry on manifolds. Again, Hill utilizes the identification that you would probably not like (with things like 'common background frames' in play).

Zhigang Suo's picture

Dear Amit:  Thank you very much for this helpful discussion.  You understand me perfectly, and I believe that we agree.  I am happy with things that work well, but still wish to understand why they work.  

It might be useful to trace the steps that may justify the practice.

Say we adopt the definition of vector space in linear algebra.  This definition invoves a number field K and a set V, along with two operations:  the addition of two elements in V, and the multiplication of an element in K and an element in V.  What makes vector space a useful object is that a linear combination of two elements in V is yet another element in V.

We wish to apply this definition to physical objects, such as displacement and force.  We both agree, I believe, that these two objects belong to two distinct vector spaces.  A displacement belongs to one vector space U, and a force belongs to another vector space V.  By these statements we mean that a linear combination of two displacements is yet another element in U, and that a linear combination of two forces is yest another element in V. In forming linear combinations, we use elements in the number field K, taken to be the real numbers.

Why is the set of displacements a vector space?  I suppose that it is a model of our physical world.  

But our physical world does not let us form a linear combination of a displacement and a force.  If we respect both our physical world and the definition of linear space, we must say that displacements and forces belong to two distinct linear spaces.

Why is the set of forces a vector space?  We may also say that it is a model of our physical world.  

Alternatively, we can relate this physical fact to other physical facts.  For instance, we can begin with the law of the conservation of energy, and try to find a way to calculate the change in energy associated with motion in space (i.e., the displacement).  We find from experience that there is a linear map that maps a displacement in U to a scalar, the change in energy (i.e. the work).  We call this linear map the force.  Thus, the force is an element in the dual space of U.  The rest, as they say, is algebra.

Thank you so much for uploading Don Carlson's class notes.  They brought back warm memories of the man, his scholarship and kindness.  I have added a comment to your post.  I have downloaded the notes, and will learn from them.

Amit Acharya's picture

Hi Zhigang: For me, the work, force, displacement example at least provides a way of seeing why the conventional mechanics interpretation  works out because the underlying vector space has an inner-product and it is easy to see why the force can be associated with a vector in the same space as u. (And even physically, I for one can feel the direction and magnitude of a shove more immediately than something from an abstract dual space to displacements contacting me :). 

However, the other question of why conventional mechanics gets away with treating position vectors, displacements, velocities, area vectors, tractions etc. as all belonging to the same vector space (when physically we cannot really add displacements and velocites, e.g.), does not seem to get addressed by representation theorem duality.... This is even before we get to linear transformations......It would be good to know if someone has sat down and worked out the logic carefully.....It must have to do with dimensional analysis too, I guess....

Zhigang Suo's picture

Dear Amit:  Thank you so much for keeping us focused on the question "why conventional mechanics gets away..."  Here are some thoughts stimulated by this focus.

Model our physical space as a vector space.  Our physical space is not a vector space as defined in linear algebra.  However, once we pick a point in our physical space as the origin, we can model the set of all points in the physical space as a three-dimensional vector space on the field of real numbers.  An element of this vector space represents the position of one point in the physical space relative to the origin.  

We add one more structure to this vector space:  an inner product.  Thus, we model our physical space by a 3D Euclidean space.  Let's call this vector space U.   

Many people have discussed this model of physical space.  We know that this model is not an exact representation of our physical space, but we will accept the model as an approximate representation.      

A basis of the vector space.  Because U is a vector space, a linear combination of any two elements in U is still an element in U.  We can choose any three linearly independent elements in U as a basis.  We can use the inner-product to represent length and angle.  Furthermore, we can choose three orthonormal elements in the vector space as a basis.

Force.  We have discussed the force.  If we are willing to start with the law of the conservation of energy, then force is a way to calculate the change in energy associated with movement in the physical space.  Specifically, force is a linear map that maps a difference of two positions to a difference in energy.  (The map is linear if the change is smooth and the two positions are close to each other.  The Taylor expansion.)  

Energy is a scalar, and the difference of two positions is a vector, an element in U.  Thus, force is an element in another vector space, V.  But U and V are dual spaces.  As explained in this Wikipedia page dual spaces, we can choose a dual basis for V.  Because U is an Euclidean space, its dual space V is also an Euclidean space.  Furthermore, we can choose the basis of V to coincide with the basis of U.

Velocity.  Velocity is the diference between the positions divided by the change in time.  Because time is a scalar, the set of velosities is another Euclidean space.  We can choose the basis of this space to coincide with the basis of U.

Electric field. The electric field is a linear map that maps the difference in position to change in electric potential.  Lke force, the electric field is an element of a dual space of U.

Any other vector?  All the above examples relate to the position space U.  How about a vector unrelated to the physical space?  Say we have yet another three-dimensional vector space M.  A linear combination of any two elements in M is yet another element in M.  We do not wish to exploit any relation between M and U, except that they are both 3D vector spaces.

We can select three linearly independent elements in M as a basis of M.  There is no way for us to relate the basis of M to the basis of U.  

What is the question then?  Here is a possibe one.  Suppose that we have discovered a linear map A that maps an element in U to an element in M.  Once we select a basis for U and a basis for M, we can represent A as a matrix of 9 components.  If we seperately change the two bases, the components of A will change accordingly, like a two-point tensor.  If we wish, we can always choose to change the two bases in the same way, by using a single matrix for transformation.  This way, the components of A will change like a regular tensor.

True stress.  In this veiw, even the true stress is a two-point tensor, becasue the true stress is a linear map that maps an area vector to a force acting on the area.  The area vector belongs to a vector space, and the force belongs to another vector space. That stress is a linear map is established by the balance of forces, and has nothing to do with how we choose basis for either vector space.  By convention, we choose to make the bases of the two vector spaces coincide.  But we can also act up, and choose different bases for the two vector spaces.  The choice of bases has no physical consequence.

Summary.  The conventional mechanics gets away with it by a convention:  making bases of different vector spaces transform in a similar way.    

Amit Acharya's picture

Zhigang: Under 'Velocity' you mentioned "We can choose the basis of this space to coincide with the basis of U." Since there is no unique basis to choose, and in any case velocities are still velocities and positions, positions, in the context of this discussion in my opinion this case is also best left as the general M case that you discuss afterwards.

The linear map A that you try to describe is an isomorphism that I mentioned in an earlier comment between two finite dimensional vector spaces of equal dimension. For me, first such a physically meaningful mapping would have to be discovered for each pair of spaces. Otherwise, there are infinitely many that can be constructed, and since physics does not care for convention, i.e. physical results have to be invariant w.r.t. choice of convention, it needs to be shown that your choice of a map A between say the space of velocities and position vectors (A is independent of bases in either space), and my choice of another map A between the same spaces keep all the mechanics we want to do invariant...Alternatively, I cannot see a physically meaningful linear map that associates each possible position vector to a corresponding velocity vector....and even if I did, I am not sure to what extent you or a third person would accept that is the only choice to work with. So it seems like proving the invariance with respect to different choices of the mapping A might be the way to go, and this needs some careful reasoning, which I am probably not doing typing on imechanica, so this tells me that if I should talk about this any further, I should better get my act together and do some serious thinking in solitude and then get back to say something if I have anything meaningful further to say...

 Until then, with my best regards,

- Amit

arash_yavari's picture

Dear Zhigang and Amit:

Thanks for this interesting discussion. Just some thoughts:

Let us consider a very simple example. A particle moving in the Euclidean three space. Position vector x(t) at any instant of time t can be written with respect to the canonical Cartesian basis vectors {i,j,k}. If you now differentiate x(t) with respect to time you get the velocity v(t) that has components with respect to the same basis {i,j,k}. We think of both the position vector and velocity as vectors living in the same linear space. But with the understanding that we cannot add a position vector and a velocity vector as they have different physical dimensions.

Note also that when curvilinear coordinates are used the natural bases would have different physical dimensions and consequently components of a vector may not have the right physical dimensions, e.g. in spherical coordinates. Here, one would have to use the so-called physical components.

If this particle moves in a curved space then position vector is not even defined. However, velocity is defined and at any given time t it lies in the tangent space to the ambient space at the point x(t). So, in this case one cannot even pose the question of adding "position" and velocity vectors.

Let us call the linear space of velocities at a point x, V. Using rate of change of energy or power, velocity and force are dual to each other (similarly force and displacement are dual to each other as Zhigang mentioned). Forces live in the dual space of V, i.e. V*. This is the space of one-forms or covectors. Choosing a basis for V, one has a natural dual basis for V*. These, in general, have different physical dimensions. There is a natural pairing between elements of V and V* and this natural pairing does not require an inner product (or metric). An inner product (metric) induces an isomorphism between V and V*, i.e. one can identify the two linear spaces. Note that, in general, this isometry changes from point to point.

Given two linear spaces V and W an isomorphism is a one-to-one and onto linear map. If you specify the map on a basis for V then the map is uniquely defined. The two vector spaces being isomorphic doesn't mean you can add a vector in V to a vector in W. In this sense, identifying the linear space of forces with that of velocities doesn't mean you can add the two.

I think Cauchy stress is a tensor in the sense of tensors in the notes of Prof. Carlson using the isomorphy between vectors and covectors. One doesn't even need to worry about identifying the linear spaces of forces and velocities. Cauchy stress can be simply represented using a basis of covectors (and is obviously symmetric). However, again, I agree with Zhigang that for mechanics applications one should work with multilinear maps on a collection of linear spaces that are not necessarily identical. The obvious examples are deformation gradient and the first Piola-Kirchhoff stress.


Amit Acharya's picture

Hi Arash: A few remarks on your comments:

1. "We think of both the position vector and velocity as vectors living in
the same linear space. But with the understanding that we cannot add a
position vector and a velocity vector as they have different physical

This is the crux of the problem, as it sounds like a contradiction in terms. Ideally, we would like to be able to say that if two objects live in the same linear space then they absolutely should be'addable' and if not, then they do not belong to the same linear space.

2. "If this particle moves in a curved space then position vector is not even defined."

I get the point you are making and the context, but just to play devil's advocate, as long there is a good, physical embedding space, it all works out just fine. E.g a 2-d shell in 3-d space - of course, we do very good mechanics using position vectors on the shell.

 3. "Given two linear spaces V and W an isomorphism is a one-to-one and onto
linear map. If you specify the map on a basis for V then the map is
uniquely defined. The two vector spaces being isomorphic doesn't mean
you can add a vector in V to a vector in W. In this sense, identifying
the linear space of forces with that of velocities doesn't mean you can
add the two."

For any linear transformation (on a finite dimensional space at least), whether an isomorphism or not, knowing its action on a basis completely determines it, obviously. However, this property of all linear transformations (on fd spaces) obviously does not uniquely pick any one of them.

The goal in utilizing an isomorphism would not be to add vectors from the different spaces. The goal would be to work with only one space and hopefully be able to do mechanics relevant to the other through this space, utilizing the identification afforded by the isomorphism.

4. "However, again, I agree with Zhigang that for mechanics applications one
should work with multilinear maps on a collection of linear spaces that
are not necessarily identical. The obvious examples are deformation
gradient and the first Piola-Kirchhoff stress."

Since this is just an opinion, let me share mine from practical experience. I have followed beautiful, nonlinear shell theory (of the Antman, Fox-Simo, etc. variety), done some of that theory, and also numerically implemented some without ever thinking of multilinear maps. Anyone who as done shell theory will immediately realize that you have to deal with the deformation gradient on a shell by considering the tangent spaces at corresponding points on reference and target shell as fundamentally different, but I do not see this as any reason for making the multilinear map point of view as essential for mechanics application......It is what it is - just a point of view.

arash_yavari's picture

Dear Amit:

Thanks for your remarks. I think the main one I'd like to understand is the first remark (I pretty much agree with your other comments). I'm not sure if this is the solution but one may be able to think of two copies of the same linear space. In other words, position vector and velocity vector live on two copies of the same linear space. This would mean that you can't even mathematically add vectors from the two copies.

This idea of different copies of the same linear space you clearly see in differential geometry. Given an n-manifold, tangent space at any point is an n-dimensional linear space. You have different copies of the same linear space at different points. These linear spaces are all isomorphic but you cannot add vectors from different copies of the same linear space. The extra structure that would allow you to do so is "connection". Just some thoughts of course.


Amit Acharya's picture

Hi Arash,

I of course agree with what you say about a linear connection, but that takes us into calculus on manifolds while in this discussion everything is in some sense in 'one tangent space.'

In fact, in my opinion that is where the fuss with different linear spaces really starts to matter, and one has to take a stance. For instance, suppose we fuss a lot in the linear algebra and then when we have to do calculus we adopt balance of linear momentum and start adding traction vectors from different tangent spaces (the latter I am perfectly happy to do and have bought into, and I would not fuss with the linear algebra either), then what was the initial fuss while doing algebra about? I know of your work on covariantization of elasticity, so you seem to 'fuss consistently,' which I like - (this is a joke, take it lightly) .

Amit Acharya's picture

Hi Zhigang and Arash:

Here is a thought that I have not completely checked through for logical consistency:

One takes a linear space, of say, dimension 3, that contains objects that have no physical dimensions.

Suppose we now take a fundamental set of physical units. Then we nondimensionalize each of the elements  of each of the dimension 3 physical vector spaces of interest - space of position vectors, velocities, area vectors, tractions, etc. - appropriately. For each of these physical vector spaces, this act of nondimensionalization induces a mapping between them and the non-dimensional vector space (say, DND mapping).

We then do linear algebra and mechanics on the non-dimensional space, defining tensors etc. based on it. For interpretation of results, we go back, using the inverse of the appropriate DND mapping to the relevant vector space.

The DND mappings may be isomprhisms - I have not checked carefully. But even if they are not, it seems like this is the right physical choice. Moreover, there should be an invariance of implied mechanics over all possible choice of fundamental sets of physical units chosen to induce the DND mappings, simply by the invariance of the laws of mechanics over choices of fundamental units.

May be this is what is implied in mechanics when we lump position vectors, velocities, tractions etc. in the same vector space.

arash_yavari's picture

Hi Amit,

If I understand you correctly, pretty much you're saying let's look at different copies of the same linear space (using your DND mappings) and each quantity of interest lives in one copy. What do you think?


Amit Acharya's picture

Hi Arash,

Yes, that's what I am saying. Of course, the main issue in this discussion that took a bit of thinking was the manner in which the copies should be constructed as there are infinitely many ways of doing so and one wants to find the way that is consistent with what we routinely do in continuum mechanics, the latter of course being sensible, given the weight of its results.

I find the the DND mappings interesting in that by dimensional consistency they link the various copies - they cannot be generated from completely arbitrary isomorphisms, say.

Zhigang Suo's picture

I find an old iMechanica thread on two-point tensors, such as deformation gradient and nominal stress.  In that thread, I just added a new comment to describe how I teach the subject.

Konstantin Volokh's picture

Dear Zhigang,

Map is an action. Nobody has to to define tensor as an action. It can be defined as an object - analogously to vector. No maps no problems :)


Zhigang Suo's picture

Maps are objects as well.  In linear algebra, tensors are defined as linear maps.  See a comment below:  linear algebra done wrong.

More importantly, we mechanicians have been using linear maps to define tensors all the time.  Here are familiar examples:

  • Stress is a linear map that maps an area vector to the force acting on the area.
  • Deformation gradient is a linear map that maps a segment of staight line of material particles from the reference state to the current state.

Maybe you and I use different words, but we mean the same things.  I have descibed these definitions in my notes on finite deformaton.   

Konstantin Volokh's picture

Not exactly. You define maps by their actions. Objects are defined by their properties. You do not need to map vector to vector in order to define tensor. You may define tensor by the transformation properties of its components (as Levi-Civita did). Or you may define tensor as a linear combination of dyads, tryads etc. They can be used for maps but they do not need maps for their definition.

Zhigang Suo's picture

Dear Kosta, Very well then.  Please outline how you show that stress is a tensor. 

Konstantin Volokh's picture

Zhigang, Cauchy stress is defined as a sum of three dyadic products of vectors - see Eq. (3.21) of my lecture notes on Mechanics of Soft Active Materials.Thus, it is a tensor by construction.

Zhigang Suo's picture

Who gives the authority to Cauchy to define stress as a sum of dyads?

Zhigang Suo's picture

Dear Kosta:  Your notes on the Mechanics of Soft Materials are beautiful.  Writing to you today brought back the memory of our first meeting, in August 2006, at Brown University, at the symposium celebrating the 60th birthday of Alan Needleman and Viggo Tvergaard.  Or was that our first meeting?  In any case, you were already deep in the mechanics of soft materials, and I was still a hardy.  We were on separate recruiting missions.  You were persuading me that the mechanics of soft materials was coming.  I was persuading you that iMechanica was coming.

It turned out neither of us needed much persuasion.  You became iMechanica User 134, and a very vocal one.  You gave me a hard time, on iMechanica, all in good spirit, on my first paper on the mechanics of soft materials.  I posted the paper on 1 January 2027, and you made your first comment on the paper on the second day.  Your comments made great impression on me and my students.  Thank you.

Now back to the question, Who gave the authority to Cauchy to define stress as a tensor, as we call it today?

The answer, it seems to me, is Euclid, Newton, and whoever invented the notion of tensor.  The statement "stress is a tensor" results from a combination of geometry, mechanics, and algebra.

In your notes, by force balance and geometry, you reach equation (3.17).  The equation says that the force acting on an area is linear in the area vector.  

The rest, as they say, is algebra.  The area vector is an element in a vector space.  The force is an element in another vector space.   Equation (3.17) says that a linear map exists, mapping an area vector to the force acting on the area. 

In linear algebra, we call a linear map between vector spaces a tensor.

In mechanics, we name this particular map the stress.

Incidentally, the existence of a linear map between the area vector and the force can be established without using any basis.  I gave such a derivation in my notes on finite deformation.  The derivation applies to both the nominal stress and the true stress.

Happy new year!  May we have many more happy memories of talkig to each other in many more new years. 

Konstantin Volokh's picture

Dear Zhigang,

Thank you for refreshing good memories :)

Happy New Year to you too!

Let us map old discussions into new year.

I am a big fan of iMechanica - I learn from it (not only your students do that). I have a feeling that you are becoming a mathematician. This is a dangerous process and I will try to slow it down in May :)

Zhigang Suo's picture

My colleague, Michael Brenner, is spreading the wisdom that

  1. The applied mathematician does not create mathematics just as the historian does not create history.
  2. We are all applied mathematicians.
N. Sukumar's picture

Zhigang:  I have found the book by Strang (one of the books listed in your link) to be an excellent introduction to linear (matrix) algebra. It is readily accessible by any undergraduate/graduate in engineering. Even though you might be looking for a more advanced textbook (links to tensor algebra and mechanics), I thought of putting in a plug for Strang's book. Besides the necessary math, the distinguishing feature of the book is a clear presentation, with geometry as an aid, of the main concepts (role of row- and column-spaces) in linear algebra. Armed with the know-how from this book, a student will be better-prepared to grasp more advanced topics in linear algebra.

Zhigang Suo's picture

Thank you Suku.  I heard about the book by Strang, but have not studied it.  Students rave about vidoes of his MIT lectures.

Harold S. Park's picture

I would like to second Suku's endorsement of Strang.  It's a very clearly written text written in a style that can be accessed by both engineers and mathematicians, I believe.  As an aside, Strang was one of the forerunners in doing mathematical analysis of the finite element method, so those researchers interested in that subject can become familiar with Strang's style via his linear algebra textbook.

Kaushik Dayal's picture

I have read the books by Naylor & Sell and Jim Knowles mentioned above.  I liked both of them.  In addition, I find Horn and Johnson's book very useful.  However, it is not an introductory book.  Peter Lax has a Linear Algebra text that I have not read, but will be good if one can generalize from the clear style of his Functional Analysis text.

Of those that I have listed above, only Knowles' book discusses tensors.  Mort Gurtin's continuum mechanics books have much of what is required in continuum mechanics for tensor algebra and calculus.

Kaushik Dayal 

Zhigang Suo's picture

Dear Kaushik:  Thank you for the suggestion.  The book by Horn and Johnson has been on my shelf for many years, but I have never got around to study it!  Your recommendation gets me intereted in the book now.  After reading your comment, I find that the book has a second edition recently.  

Konstantin Volokh's picture

Strang is the eye-opener. He gives intuitive geometric view of the subject. Mathematicians do not like him but I strongly recommend Strang for any physicist or engineer.

Zhigang Suo's picture

In a short note, Strang commented on our allocation of time of education:  too much calculus, too little linear algebra.

In 1953, the French mathematician Jean Dieudonne wrote in a book review:

"Linear algebra is now universally recognized as perhaps the most important tool of the modern mathematician; its concepts and methods, moreover, when properly reduced to their essential features, are among the simplest and most straightforward imaginable.  Nevertheless, it is still not uncommon to find graduate students who are totally unfamiliar with some of the fundamental notions of linear algebra, such as, for instance, the theory of duality."

I wonder if anything has happened in the last 60 years that may cause him to rewrite the above sentences. 

N. Sukumar's picture

Definitely.  For a mechanician, I think a solid foundation in linear algebra and (convex) optimization is invaluable.  With a clear understanding of the concepts in matrix algebra that Strang lists, understanding finite-dimensional function spaces is eased.  From Strang's book, I found that thinking of Ax as a linear combination of the columns (row-picture is the norm) is the main take-home message. His online lectures are very clear.  Variational methods in mechanics are often solved in a finite-dimensional setting (e.g., Galerkin methods) with a set of linearly independent basis functions. The end result is a linear system of equations Ax = b that has to be solved;  since A is formed via a suitable `inner product' of basis functions,  tools from linear algebra can be used to provide a mathematically sound means to construct A.  The recently proposed virtual element method adopts this viewpoint.

Konstantin Volokh's picture

Alternatives to Strang:
Brown - A second Course in Linear Algebra
Hoffman & Kunze - Linear Algebra

Julian J. Rimoli's picture


I usually recommend my Ph.D. students to take Arash’s course on Applied Math. I also developed a self-contained class on Fundamentals of Solid Mechanics aiming to expose first year graduate students to a broad range of topics in the area within a unified and modern framework. Necessarily, I have to cover some basic tensor algebra at the beginning of the course. I found that the book by Tadmor et al on Continuum Mechanics  (Chapter 2) does an excellent job at introducing the topic in a way that is accessible for engineers without sacrificing much rigor.


Julian J. Rimoli


Shailendra's picture

Dear Zhigang,

Thanks for bringing up an important topic. I teach two continuum mechanics courses at the undergraduate level (sophomore and junior) and need to balance between the rigor of linear algebra, tensor calculus and the applied aspects of continuum mechanics. J.N.Reddy's book "Principles of Continuum Mechanics" is written from the viewpoint of undergraduate students.

I agree with Julian's comment above on the book by Tadmor, Miller and Elliott; it has the necessary basics on tensors written in a lucid manner. Over the past couple of years, I have been working on customized class-notes for these courses that rely on several sources.
Some references that I have found extremely useful are: 

1. Lecture notes by Rohan Abeyaratne:

2. Introduction to the Mechanics of a Continuous Medium (Malvern)

3. The Mechanics and Thermodynamics of Continua (Gurtin, Fried and Anand)

Ray Bowen's elaborate notes on Linear Algebra and Continuum Mechanics are also available online at  


Dian Yang's picture

Dear Prof Suo,

As an math major undergraduate, I used the text book <Matrix Analysis> by Roger A. Horn and Charles R. Johnson. I want to just briefly mention this book to complement the great set of books people have already suggested. This book is a little bit heavy on math, but it is an excellent second book on linear algebra. The definitions are clear, discussions are succinct, and proofs are complete, making this book very good for reference.

It has detailed discussion on the exact conditions on diagonalization, co-diagonalization, triagularization, and various matrix factorizations - making it very useful for deriving general theories in mechanics, and physics in general.

Zhigang Suo's picture

I have just added a comment titled "recipe to make new tensors".  I suppose this recipe is how mechanicians routinely "discover" or "create" new tensors.

Jing_Yang's picture

Dear Prof. Suo,

As the best of my knowledge, this book has a wide audience for beginners from math and physics departments in our university.

Most linear algebra textbook written for engineering students may neglect the mathematical aspects of linear space. Although the book "Linear Algebra Done Right" is only an introductory book, it covers and describes all the basic knowledge of finite vector space completely with abract formulation and concrete examples.

The elegant arrangement of the book, namely, abract formulation, proof in conjunction with concrete examples make it  accessible even to students with only knowledge of basic caculus. The electronic copy could be downloaded by the following link.




Zhigang Suo's picture

This is the title of the book by the mathematician Sergei Treil, of Bowen University.  

He noted that the book is for a first course on linear algebra for mathematically advanced students.  He posted the book online.  For some time I have had the 2004 edition, and have enjoyed the book:  the beautiful layout, careful sequencing of ideas, and masterful pedagogy.  It is a short book, ~200 pages.

This morning I went to his website, and found a 2009 edition!  In this new edition, he has added a chapter on dual spaces and tensors.

Here is how he defines tensors:

  1. V1, V2,..., Vp, V are vector spaces over a field F.
  2. A multilinear map that maps elements in V1, V2,...,Vp to an element in V.
  3. The multilinear map is called a tensor if V is a scalar over the field F. 

Not sure why he does not stop at 2, and simply defines tensor as a multilinear map from vector spaces to yet another vector space.  Noneheless a few pages later in the book, he did show that a linear map from one vector space to another vector is also a tensor.

I believe that mechanicians have been routinely "discovering" or "creating" new tensors from known tensors, as I commented elsewhere:  recipe to make new tensors.  We essentially stop at 2.

Please also see the discussion above among Arash, Amit, and me.

What do you think? 

Amit Acharya's picture

Hi Zhigang: I haven't had the time to take a look at the book you mention, but here is a guess as to why it makes a lot of sense to define whatever one needs (e.g. 2nd order tensors, higher order tensors), especially when doing mechanics - it is to ultimately to do the mechanics one wants to do. If 2nd order tensors are all one needs, one does that. If higher order tensors are needed (as from the time of the Brothers Cosserat, at least, in continuum mechanics, through Green-Rivlin, Noll, Truesdell etc.; of course linear elasticity came even before that), one uses a logically satisfactory definition for that, and then gets on with life. And, based on these examples, I wouldn't say mechanicians stop at order 2.

As for "routinely "creating"/"discovering" new tensors" - I feel it is important to add to your statement that while any mechanician with an iota of basic training and good sense appreciates equivalences between disparate definitions as a deeper understanding of his/her subject, none in this class think that coming up with such definitions is an achievement in its own right or that one particular definition among equivalent definitions is somehow better. Moreover, they are also completely agnostic and mercenary in their approach to such definitions - they learn all of them as well as they can to deploy (all of) them effectively as particular situations demand.

And, a final note - let's not forget the purely component-transformation-rule based approach for introducing tensors that we have not discussed at all. Again, that is an equivalent definition, and Einstein would be very mad with us - or think we were really ignorant - if we were to tell him that the multilinear approach of introducing tensors is really the way to think about tensors and their applications to mechanics.

Zhigang Suo's picture

Dear Amit:  Sorry for being unclear.  When I wrote, "We essentially stop at 2", I meant to say stop at Point 2 in the definition listed in the comment.   That is, we essentially stop at the following point:  tensor is a multiliear map that maps elements in V1, V2,..., Vp to an element in V.  This definition, of course, let us go beyond second-order tensor.

I believe that the definition of tensor in Treil's book is broader than what we normally use.  For example, this definition will allow a tensor with compoenents forming a rectangular matrix, instead of being limited to a squre matrix.  That is, this definition is not equivalent to what we commonly adopt.

I like your spirit:  everything goes, so long as we do good mechanics. 

arash_yavari's picture

Dear Zhigang:

I haven't looked at this book but I don't think we're restricted to work with linear spaces of the same dimension the way we define tensors. For example, deformation gradient for a shell is a linear map from a two-dimensional vector space to a three-dimensional vector space. So, it is a two-point tensor and its matrix representation is not a square matrix.

I agree with Amit that one shouldn't worry too much about what definition is better and/or more general. I would say one should use what is appropriate for the application of interest.


Zhigang Suo's picture

Dear Arash:  Thank you so much for the comment.  I believe that you and I are both happy with defining tensor as the following

  1. V1, V2,..., Vp, V are vector spaces over a field F.
  2. A tensor is a multilinear map that maps elements in V1, V2,...,Vp to an element in V.

I don't believe that this definition is commonly adopted among mechanictians.

Now this statement is a socialogical one, not a scientific one.  The statement can be amply confirmed by a look at existing textbooks.  For example Tadmor-Miller-Elliot (p.36) restrict V1, V2,...Vp to be vector spaces of the same dimension.  Their definition is equivalent to that in the Carlson notes (Definition A4.1.1).  One can go on to list many textbooks in mechanics that give definition equivalent to that in Tadmor-Miller-Elliot. 

Are you aware of any textbook in mechanics that adopts the definition above (Point 1 and 2)?  They must be a very small minority.  That is what I mean that this definition is not commonly adopted by mechanicians.

Is this definition (Point 1 and 2 above) too general for our basic needs?  That is debatable.  But this much is sure.  As you and I discussed before, the definition comfortably accomodates the two most basic tensors in continuum mechanics:  stress and deformation gradient.  Let me list our steps once more (for people who wish to see more details, please take a look at my notes on finite deformation):  

A special case of the definition (Point 1 and 2).  A linear map that maps an element in one vector space to an element in another vector space is a tensor.

Stress.  By balancing forces, we find a linear map that maps an area vector to the force acting on the area.  The area vector is an element in one vector space, and the force is an element in another vector space.  We call this linear map the stress, and it fits our definition of tensor.  

(Incidentally, here I use area vector, rather than unit vector normal to the area, because the latter does not form a vector space:  a linear combination of two unit vectors is in general not another unit vector.)  

Deformation gradient.  By the geometry of homogeneous deformation, we find a linear map that maps a segment of a straight line of material particles from the reference state to the current state.  The segment in the reference state is an element in one vector space, and the segment in the current state is an element in another vector space.  We call this linear map the deformation gradient, and it fits our definition of tensor.

By contrast, we have to do some dances to fit the two objects to the conventional definition of tensor.  Why should we?  Again, this is socialogical question, not a scientific one. 

arash_yavari's picture

Dear Zhigang:

The book by R.W. Ogden (Non-Linear Elastic Deformations) defines tensors very similarly to what has been done in Carlson's notes (page 43). Let me also say that this is an excellent book that focuses on finding exact solutions in nonlinear elasticity.

Marsden and Hughes (Mathematical Foundations of Elasticity) differentiate between tensors (page 65) and two-point tensors (page 70). I think they did not define anything more general as they didn't find much use for it in the context of elasticity.

Green and Zerna (Theoretical Elasticity) define tensors in terms of coordinate transformations and how tensor components change under such transformations (Amit already mentioned this).


Zhigang Suo's picture

Dear Arash:  Thank you very much.  It is fun to make a list of definitions of tensors in textbooks of continuum mechanics.  Of course, the authors may have more sophistictated ideas than they put in their books.  We are doing a phenomenological search.  What are in the textbooks are a reasonable proxy of what we teach to our students, I suppose.

I have looked at the following books: 

  • Bowen  (p.277) The same as Ogden.
  • Gurtin-Fried-Anand (p.9).  "We use the tensor as a synonym for the phrase "linear transformation from V to V"." 
  • Antman  (p.400).  The same as above.
  • Truesdel-Noll  (p.15).  The same as above.
  • Holzapfel  (p.9).  The same as above.
  • Chadwick  (p.16).  The same as above  

Of all the books you and I have looked at so far, only Marsden-Hughes allows mappings between different vector spaces.

Amit Acharya's picture

Hi Arash,

Here's how I see it:

The tangent map for either the reference or the deformed shell goes from 2-d to a 3-d space; thus  the matrix representation is rectangular as you say.

The deformation gradient goes from a 2-d tangent space to a different 2-d tangent space (vectors in one in general do not lie within the linear span of a basis for the other). Thus its matrix is square 2x2, the 'canonical' representation in convected coordinates being the 2x2 identity matrix.

Did I miss anything?

Amit Acharya's picture

Zhigang: This is in response to a statement you made.

A third-order tensor on a 3-d vector space V_3 in the direct formulation of continuum mechanics (a la Carlson, say) is a linear transformation from 3-d space to the 3x3 = 9 dimensional space of second order tensors on V_3. Thus its representation on bases yields a rectangular 9 by 3 matrix. Similarly the transpose of this tensor is a linear transformation from the 9 dimensional space of second order tensors to the 3-d space - that yields a 3 x 9 matrix.

So what we do in continuum mechanics very much results in rectangular matrices when required and quite naturally (unlike what you claimed to be the case), but obviously these are trivial details not to be fussed over.

For some serious actual use of 3rd order tensors in continuum mechanics with the direct formulation definition:

Teng Li's picture

All, thanks for a fruitful cascade of discussion, which I follow with great interest. I've updated the Lecture Notes page of iMechanica to include downloadable lecture notes on linear algebra alluded in the discussions under the "Mathematics for Mechanics" section.

Zhigang Suo's picture

This thread has become very interesting and very long.  I have another issue on which I'd like to get your opinion.  It's about scalars.  I have just posted a note titled Scalar Done Wrong.

Subscribe to Comments for "Textbook on linear algebra"

Recent comments

More comments


Subscribe to Syndicate