{"id":365,"date":"2020-08-07T21:47:48","date_gmt":"2020-08-07T21:47:48","guid":{"rendered":"http:\/\/nikolapacekvetnic.rs\/?p=365"},"modified":"2020-08-07T21:47:50","modified_gmt":"2020-08-07T21:47:50","slug":"transfer-learning-finding-learning-rate-and-more","status":"publish","type":"post","link":"http:\/\/nikolapacekvetnic.rs\/?p=365","title":{"rendered":"Transfer Learning, Finding Learning Rate, and More"},"content":{"rendered":"\n<p><em>Training a neural network from scratch is unnecessary in a situation when pretrained models are readily available. This notebook showcases loading a pretrained model directly from pytorch.models subpackage, its subsequent re-training for the specific problem at hand, a method used to determine the optimal learning rate, ways of defining custom transforms and finally assembling several models into an ensemble.<\/em><\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"168\" src=\"http:\/\/nikolapacekvetnic.rs\/wp-content\/uploads\/2020\/08\/main-qimg-fd9f8d4c4a8c1a857c0523c6d32d9c4d.jpg\" alt=\"PyTorch_Article\" class=\"wp-image-363\" srcset=\"http:\/\/nikolapacekvetnic.rs\/wp-content\/uploads\/2020\/08\/main-qimg-fd9f8d4c4a8c1a857c0523c6d32d9c4d.jpg 600w, http:\/\/nikolapacekvetnic.rs\/wp-content\/uploads\/2020\/08\/main-qimg-fd9f8d4c4a8c1a857c0523c6d32d9c4d-300x84.jpg 300w, http:\/\/nikolapacekvetnic.rs\/wp-content\/uploads\/2020\/08\/main-qimg-fd9f8d4c4a8c1a857c0523c6d32d9c4d-210x59.jpg 210w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n\n<p>Once again, the examples are based on Chapter 4 of Ian Pointer\u2019s <a href=\"https:\/\/www.oreilly.com\/library\/view\/programming-pytorch-for\/9781492045342\/\" target=\"_blank\" rel=\"noreferrer noopener\">\u201cProgramming PyTorch for Deep Learning\u201d<\/a>, released by O\u2019Reilly in 2019.<\/p>\n\n\n\n<p>Jupyter notebook of the model may be found on GitHub: <a href=\"https:\/\/tinyurl.com\/yyzth69w\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/tinyurl.com\/yyzth69w<\/a><\/p>\n\n\n\n<p>Also, the same notebook is available on Google Colab (where it can be tested it against the GPU available there): <a href=\"https:\/\/tinyurl.com\/y4rlnqfs\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/tinyurl.com\/y4rlnqfs<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Training a neural network from scratch is unnecessary in a situation when pretrained models are readily available. This notebook showcases loading a pretrained model directly from pytorch.models subpackage, its subsequent re-training for the specific problem at hand, a method used to determine the optimal learning rate, ways of defining custom transforms and finally assembling several [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":348,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55,56],"tags":[69,67,79,70,68,78,57,16,75,76,82,66,81,63,65,74,77,58,59,83,64,80,73],"class_list":["post-365","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-code","category-deep-learning","tag-cnn","tag-colaboratory","tag-color-space","tag-convolution","tag-convolution-neural-network","tag-custom-transforms","tag-deep-learning","tag-ensemble","tag-find-learning-rate","tag-find-lr","tag-gaussian-noise","tag-google-colab","tag-hsv","tag-image-classifier","tag-jupyter","tag-learning-rate","tag-lr","tag-neural-networks","tag-nn","tag-noise","tag-pytorch","tag-rgb","tag-transfer-learning"],"_links":{"self":[{"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=\/wp\/v2\/posts\/365","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=365"}],"version-history":[{"count":1,"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=\/wp\/v2\/posts\/365\/revisions"}],"predecessor-version":[{"id":366,"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=\/wp\/v2\/posts\/365\/revisions\/366"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=\/wp\/v2\/media\/348"}],"wp:attachment":[{"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=365"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=365"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/nikolapacekvetnic.rs\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=365"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}