Gdtyjr

Can you reject a postdoc offer after the PI has paid a large sum for flights/accommodation for your visit?

infinitive telling the purpose

How are such low op-amp input currents possible?

Rejected in 4th interview round citing insufficient years of experience

Are the terms "stab" and "staccato" synonyms?

PTIJ: Why can't I eat anything?

The bar has been raised

"One can do his homework in the library"

How much stiffer are 23c tires over 28c?

Does splitting a potentially monolithic application into several smaller ones help prevent bugs?

Best approach to update all entries in a list that is paginated?

Make a transparent 448*448 image

Fourth person (in Slavey language)

How do you like my writing?

Do I really need to have a scientific explanation for my premise?

Should I take out a loan for a friend to invest on my behalf?

How did Alan Turing break the enigma code using the hint given by the lady in the bar?

Should QA ask requirements to developers?

Making a sword in the stone, in a medieval world without magic

show this identity with trigometric

Latest web browser compatible with Windows 98

Grey hair or white hair

Built-In Shelves/Bookcases - IKEA vs Built

Is it true that real estate prices mainly go up?

What are the best books to study Neural Networks from a purely mathematical perspective?

Modern Mathematical Theory for Neural Networks, Cellular Automata, NeuroscienceTwo-layer Perceptron for XORdelay time from mathematical perspectiveReference Request: Neural Networks as Mathematical ModelsSources to learn and understand advanced probability in ML modelsHow can logicians benefit from development of artificial neural networks?Deriving diagonal approximation of Hessian in a neural networkNeural Networks and the Chain RuleWhat are some current best practices for function approximation using neural networks?What does oplus symbol ⊕ do for 2 images in Convolution Neural Networks

5

3

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

5

3

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 2 hours ago

EliEli

341

2 Answers
2

active

oldest

votes

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

Eli is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3146016%2fwhat-are-the-best-books-to-study-neural-networks-from-a-purely-mathematical-pers%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144