mirror of
https://github.com/autistic-symposium/tensorflow-for-deep-learning-py.git
synced 2025-05-10 10:45:04 -04:00
add new deep dream stuff from aws repo
This commit is contained in:
Marina von Steinkirch
parent
0d8220a392
commit
305e85e8b7
25 changed files with 30870 additions and 517 deletions
|
|
@ -1,517 +0,0 @@
|
|||
All code in this repository is licensed under Apache License 2.0.
|
||||
All images and derivatives thereof contained in this repository
|
||||
are licensed under Creative Commons License 3.0. Original photo by Mohammed Tawsif Salam,
|
||||
https://en.wikipedia.org/wiki/Sky#/media/File:Appearance_of_sky_for_weather_forecast,_Dhaka,_Bangladesh.JPG
|
||||
Full text of Apache License 2.0 and Creative Commons License 3.0 below:
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Creative Commons
|
||||
Attribution-ShareAlike 3.0 Unported
|
||||
https://creativecommons.org/licenses/by-sa/3.0/legalcode
|
||||
|
||||
CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE LEGAL
|
||||
SERVICES. DISTRIBUTION OF THIS LICENSE DOES NOT CREATE AN ATTORNEY-CLIENT
|
||||
RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS INFORMATION ON AN "AS-IS" BASIS.
|
||||
CREATIVE COMMONS MAKES NO WARRANTIES REGARDING THE INFORMATION PROVIDED, AND
|
||||
DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM ITS USE.
|
||||
License
|
||||
|
||||
THE WORK (AS DEFINED BELOW) IS PROVIDED UNDER THE TERMS OF THIS CREATIVE
|
||||
COMMONS PUBLIC LICENSE ("CCPL" OR "LICENSE"). THE WORK IS PROTECTED BY
|
||||
COPYRIGHT AND/OR OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS
|
||||
AUTHORIZED UNDER THIS LICENSE OR COPYRIGHT LAW IS PROHIBITED.
|
||||
|
||||
BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE
|
||||
BOUND BY THE TERMS OF THIS LICENSE. TO THE EXTENT THIS LICENSE MAY BE
|
||||
CONSIDERED TO BE A CONTRACT, THE LICENSOR GRANTS YOU THE RIGHTS CONTAINED HERE
|
||||
IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS.
|
||||
|
||||
1. Definitions
|
||||
|
||||
"Adaptation" means a work based upon the Work, or upon the Work and other
|
||||
pre-existing works, such as a translation, adaptation, derivative work,
|
||||
arrangement of music or other alterations of a literary or artistic work, or
|
||||
phonogram or performance and includes cinematographic adaptations or any other
|
||||
form in which the Work may be recast, transformed, or adapted including in any
|
||||
form recognizably derived from the original, except that a work that
|
||||
constitutes a Collection will not be considered an Adaptation for the purpose
|
||||
of this License. For the avoidance of doubt, where the Work is a musical work,
|
||||
performance or phonogram, the synchronization of the Work in timed-relation
|
||||
with a moving image ("synching") will be considered an Adaptation for the
|
||||
purpose of this License.
|
||||
"Collection" means a collection of literary or artistic works, such as
|
||||
encyclopedias and anthologies, or performances, phonograms or broadcasts, or
|
||||
other works or subject matter other than works listed in Section 1(f) below,
|
||||
which, by reason of the selection and arrangement of their contents, constitute
|
||||
intellectual creations, in which the Work is included in its entirety in
|
||||
unmodified form along with one or more other contributions, each constituting
|
||||
separate and independent works in themselves, which together are assembled into
|
||||
a collective whole. A work that constitutes a Collection will not be considered
|
||||
an Adaptation (as defined below) for the purposes of this License.
|
||||
"Creative Commons Compatible License" means a license that is listed at
|
||||
https://creativecommons.org/compatiblelicenses that has been approved by
|
||||
Creative Commons as being essentially equivalent to this License, including, at
|
||||
a minimum, because that license: (i) contains terms that have the same purpose,
|
||||
meaning and effect as the License Elements of this License; and, (ii)
|
||||
explicitly permits the relicensing of adaptations of works made available under
|
||||
that license under this License or a Creative Commons jurisdiction license with
|
||||
the same License Elements as this License.
|
||||
"Distribute" means to make available to the public the original and copies of
|
||||
the Work or Adaptation, as appropriate, through sale or other transfer of
|
||||
ownership.
|
||||
"License Elements" means the following high-level license attributes as
|
||||
selected by Licensor and indicated in the title of this License: Attribution,
|
||||
ShareAlike.
|
||||
"Licensor" means the individual, individuals, entity or entities that offer(s)
|
||||
the Work under the terms of this License.
|
||||
"Original Author" means, in the case of a literary or artistic work, the
|
||||
individual, individuals, entity or entities who created the Work or if no
|
||||
individual or entity can be identified, the publisher; and in addition (i) in
|
||||
the case of a performance the actors, singers, musicians, dancers, and other
|
||||
persons who act, sing, deliver, declaim, play in, interpret or otherwise
|
||||
perform literary or artistic works or expressions of folklore; (ii) in the case
|
||||
of a phonogram the producer being the person or legal entity who first fixes
|
||||
the sounds of a performance or other sounds; and, (iii) in the case of
|
||||
broadcasts, the organization that transmits the broadcast.
|
||||
"Work" means the literary and/or artistic work offered under the terms of this
|
||||
License including without limitation any production in the literary, scientific
|
||||
and artistic domain, whatever may be the mode or form of its expression
|
||||
including digital form, such as a book, pamphlet and other writing; a lecture,
|
||||
address, sermon or other work of the same nature; a dramatic or
|
||||
dramatico-musical work; a choreographic work or entertainment in dumb show; a
|
||||
musical composition with or without words; a cinematographic work to which are
|
||||
assimilated works expressed by a process analogous to cinematography; a work of
|
||||
drawing, painting, architecture, sculpture, engraving or lithography; a
|
||||
photographic work to which are assimilated works expressed by a process
|
||||
analogous to photography; a work of applied art; an illustration, map, plan,
|
||||
sketch or three-dimensional work relative to geography, topography,
|
||||
architecture or science; a performance; a broadcast; a phonogram; a compilation
|
||||
of data to the extent it is protected as a copyrightable work; or a work
|
||||
performed by a variety or circus performer to the extent it is not otherwise
|
||||
considered a literary or artistic work.
|
||||
"You" means an individual or entity exercising rights under this License who
|
||||
has not previously violated the terms of this License with respect to the Work,
|
||||
or who has received express permission from the Licensor to exercise rights
|
||||
under this License despite a previous violation.
|
||||
"Publicly Perform" means to perform public recitations of the Work and to
|
||||
communicate to the public those public recitations, by any means or process,
|
||||
including by wire or wireless means or public digital performances; to make
|
||||
available to the public Works in such a way that members of the public may
|
||||
access these Works from a place and at a place individually chosen by them; to
|
||||
perform the Work to the public by any means or process and the communication to
|
||||
the public of the performances of the Work, including by public digital
|
||||
performance; to broadcast and rebroadcast the Work by any means including
|
||||
signs, sounds or images.
|
||||
"Reproduce" means to make copies of the Work by any means including without
|
||||
limitation by sound or visual recordings and the right of fixation and
|
||||
reproducing fixations of the Work, including storage of a protected performance
|
||||
or phonogram in digital form or other electronic medium.
|
||||
2. Fair Dealing Rights. Nothing in this License is intended to reduce, limit,
|
||||
or restrict any uses free from copyright or rights arising from limitations or
|
||||
exceptions that are provided for in connection with the copyright protection
|
||||
under copyright law or other applicable laws.
|
||||
|
||||
3. License Grant. Subject to the terms and conditions of this License,
|
||||
Licensor hereby grants You a worldwide, royalty-free, non-exclusive, perpetual
|
||||
(for the duration of the applicable copyright) license to exercise the rights
|
||||
in the Work as stated below:
|
||||
|
||||
to Reproduce the Work, to incorporate the Work into one or more Collections,
|
||||
and to Reproduce the Work as incorporated in the Collections;
|
||||
to create and Reproduce Adaptations provided that any such Adaptation,
|
||||
including any translation in any medium, takes reasonable steps to clearly
|
||||
label, demarcate or otherwise identify that changes were made to the original
|
||||
Work. For example, a translation could be marked "The original work was
|
||||
translated from English to Spanish," or a modification could indicate "The
|
||||
original work has been modified.";
|
||||
to Distribute and Publicly Perform the Work including as incorporated in
|
||||
Collections; and,
|
||||
to Distribute and Publicly Perform Adaptations.
|
||||
For the avoidance of doubt:
|
||||
|
||||
Non-waivable Compulsory License Schemes. In those jurisdictions in which the
|
||||
right to collect royalties through any statutory or compulsory licensing scheme
|
||||
cannot be waived, the Licensor reserves the exclusive right to collect such
|
||||
royalties for any exercise by You of the rights granted under this License;
|
||||
Waivable Compulsory License Schemes. In those jurisdictions in which the right
|
||||
to collect royalties through any statutory or compulsory licensing scheme can
|
||||
be waived, the Licensor waives the exclusive right to collect such royalties
|
||||
for any exercise by You of the rights granted under this License; and,
|
||||
Voluntary License Schemes. The Licensor waives the right to collect royalties,
|
||||
whether individually or, in the event that the Licensor is a member of a
|
||||
collecting society that administers voluntary licensing schemes, via that
|
||||
society, from any exercise by You of the rights granted under this License.
|
||||
The above rights may be exercised in all media and formats whether now known
|
||||
or hereafter devised. The above rights include the right to make such
|
||||
modifications as are technically necessary to exercise the rights in other
|
||||
media and formats. Subject to Section 8(f), all rights not expressly granted by
|
||||
Licensor are hereby reserved.
|
||||
|
||||
4. Restrictions. The license granted in Section 3 above is expressly made
|
||||
subject to and limited by the following restrictions:
|
||||
|
||||
You may Distribute or Publicly Perform the Work only under the terms of this
|
||||
License. You must include a copy of, or the Uniform Resource Identifier (URI)
|
||||
for, this License with every copy of the Work You Distribute or Publicly
|
||||
Perform. You may not offer or impose any terms on the Work that restrict the
|
||||
terms of this License or the ability of the recipient of the Work to exercise
|
||||
the rights granted to that recipient under the terms of the License. You may
|
||||
not sublicense the Work. You must keep intact all notices that refer to this
|
||||
License and to the disclaimer of warranties with every copy of the Work You
|
||||
Distribute or Publicly Perform. When You Distribute or Publicly Perform the
|
||||
Work, You may not impose any effective technological measures on the Work that
|
||||
restrict the ability of a recipient of the Work from You to exercise the rights
|
||||
granted to that recipient under the terms of the License. This Section 4(a)
|
||||
applies to the Work as incorporated in a Collection, but this does not require
|
||||
the Collection apart from the Work itself to be made subject to the terms of
|
||||
this License. If You create a Collection, upon notice from any Licensor You
|
||||
must, to the extent practicable, remove from the Collection any credit as
|
||||
required by Section 4(c), as requested. If You create an Adaptation, upon
|
||||
notice from any Licensor You must, to the extent practicable, remove from the
|
||||
Adaptation any credit as required by Section 4(c), as requested.
|
||||
You may Distribute or Publicly Perform an Adaptation only under the terms of:
|
||||
(i) this License; (ii) a later version of this License with the same License
|
||||
Elements as this License; (iii) a Creative Commons jurisdiction license (either
|
||||
this or a later license version) that contains the same License Elements as
|
||||
this License (e.g., Attribution-ShareAlike 3.0 US)); (iv) a Creative Commons
|
||||
Compatible License. If you license the Adaptation under one of the licenses
|
||||
mentioned in (iv), you must comply with the terms of that license. If you
|
||||
license the Adaptation under the terms of any of the licenses mentioned in (i),
|
||||
(ii) or (iii) (the "Applicable License"), you must comply with the terms of the
|
||||
Applicable License generally and the following provisions: (I) You must include
|
||||
a copy of, or the URI for, the Applicable License with every copy of each
|
||||
Adaptation You Distribute or Publicly Perform; (II) You may not offer or impose
|
||||
any terms on the Adaptation that restrict the terms of the Applicable License
|
||||
or the ability of the recipient of the Adaptation to exercise the rights
|
||||
granted to that recipient under the terms of the Applicable License; (III) You
|
||||
must keep intact all notices that refer to the Applicable License and to the
|
||||
disclaimer of warranties with every copy of the Work as included in the
|
||||
Adaptation You Distribute or Publicly Perform; (IV) when You Distribute or
|
||||
Publicly Perform the Adaptation, You may not impose any effective technological
|
||||
measures on the Adaptation that restrict the ability of a recipient of the
|
||||
Adaptation from You to exercise the rights granted to that recipient under the
|
||||
terms of the Applicable License. This Section 4(b) applies to the Adaptation as
|
||||
incorporated in a Collection, but this does not require the Collection apart
|
||||
from the Adaptation itself to be made subject to the terms of the Applicable
|
||||
License.
|
||||
If You Distribute, or Publicly Perform the Work or any Adaptations or
|
||||
Collections, You must, unless a request has been made pursuant to Section 4(a),
|
||||
keep intact all copyright notices for the Work and provide, reasonable to the
|
||||
medium or means You are utilizing: (i) the name of the Original Author (or
|
||||
pseudonym, if applicable) if supplied, and/or if the Original Author and/or
|
||||
Licensor designate another party or parties (e.g., a sponsor institute,
|
||||
publishing entity, journal) for attribution ("Attribution Parties") in
|
||||
Licensor's copyright notice, terms of service or by other reasonable means, the
|
||||
name of such party or parties; (ii) the title of the Work if supplied; (iii) to
|
||||
the extent reasonably practicable, the URI, if any, that Licensor specifies to
|
||||
be associated with the Work, unless such URI does not refer to the copyright
|
||||
notice or licensing information for the Work; and (iv) , consistent with
|
||||
Ssection 3(b), in the case of an Adaptation, a credit identifying the use of
|
||||
the Work in the Adaptation (e.g., "French translation of the Work by Original
|
||||
Author," or "Screenplay based on original Work by Original Author"). The credit
|
||||
required by this Section 4(c) may be implemented in any reasonable manner;
|
||||
provided, however, that in the case of a Adaptation or Collection, at a minimum
|
||||
such credit will appear, if a credit for all contributing authors of the
|
||||
Adaptation or Collection appears, then as part of these credits and in a manner
|
||||
at least as prominent as the credits for the other contributing authors. For
|
||||
the avoidance of doubt, You may only use the credit required by this Section
|
||||
for the purpose of attribution in the manner set out above and, by exercising
|
||||
Your rights under this License, You may not implicitly or explicitly assert or
|
||||
imply any connection with, sponsorship or endorsement by the Original Author,
|
||||
Licensor and/or Attribution Parties, as appropriate, of You or Your use of the
|
||||
Work, without the separate, express prior written permission of the Original
|
||||
Author, Licensor and/or Attribution Parties.
|
||||
Except as otherwise agreed in writing by the Licensor or as may be otherwise
|
||||
permitted by applicable law, if You Reproduce, Distribute or Publicly Perform
|
||||
the Work either by itself or as part of any Adaptations or Collections, You
|
||||
must not distort, mutilate, modify or take other derogatory action in relation
|
||||
to the Work which would be prejudicial to the Original Author's honor or
|
||||
reputation. Licensor agrees that in those jurisdictions (e.g. Japan), in which
|
||||
any exercise of the right granted in Section 3(b) of this License (the right to
|
||||
make Adaptations) would be deemed to be a distortion, mutilation, modification
|
||||
or other derogatory action prejudicial to the Original Author's honor and
|
||||
reputation, the Licensor will waive or not assert, as appropriate, this
|
||||
Section, to the fullest extent permitted by the applicable national law, to
|
||||
enable You to reasonably exercise Your right under Section 3(b) of this License
|
||||
(right to make Adaptations) but not otherwise.
|
||||
5. Representations, Warranties and Disclaimer
|
||||
|
||||
UNLESS OTHERWISE MUTUALLY AGREED TO BY THE PARTIES IN WRITING, LICENSOR OFFERS
|
||||
THE WORK AS-IS AND MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND
|
||||
CONCERNING THE WORK, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, INCLUDING,
|
||||
WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTIBILITY, FITNESS FOR A
|
||||
PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS,
|
||||
ACCURACY, OR THE PRESENCE OF ABSENCE OF ERRORS, WHETHER OR NOT DISCOVERABLE.
|
||||
SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO SUCH
|
||||
EXCLUSION MAY NOT APPLY TO YOU.
|
||||
|
||||
6. Limitation on Liability. EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW,
|
||||
IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL,
|
||||
INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS
|
||||
LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE
|
||||
POSSIBILITY OF SUCH DAMAGES.
|
||||
|
||||
7. Termination
|
||||
|
||||
This License and the rights granted hereunder will terminate automatically
|
||||
upon any breach by You of the terms of this License. Individuals or entities
|
||||
who have received Adaptations or Collections from You under this License,
|
||||
however, will not have their licenses terminated provided such individuals or
|
||||
entities remain in full compliance with those licenses. Sections 1, 2, 5, 6, 7,
|
||||
and 8 will survive any termination of this License.
|
||||
Subject to the above terms and conditions, the license granted here is
|
||||
perpetual (for the duration of the applicable copyright in the Work).
|
||||
Notwithstanding the above, Licensor reserves the right to release the Work
|
||||
under different license terms or to stop distributing the Work at any time;
|
||||
provided, however that any such election will not serve to withdraw this
|
||||
License (or any other license that has been, or is required to be, granted
|
||||
under the terms of this License), and this License will continue in full force
|
||||
and effect unless terminated as stated above.
|
||||
8. Miscellaneous
|
||||
|
||||
Each time You Distribute or Publicly Perform the Work or a Collection, the
|
||||
Licensor offers to the recipient a license to the Work on the same terms and
|
||||
conditions as the license granted to You under this License.
|
||||
Each time You Distribute or Publicly Perform an Adaptation, Licensor offers to
|
||||
the recipient a license to the original Work on the same terms and conditions
|
||||
as the license granted to You under this License.
|
||||
If any provision of this License is invalid or unenforceable under applicable
|
||||
law, it shall not affect the validity or enforceability of the remainder of the
|
||||
terms of this License, and without further action by the parties to this
|
||||
agreement, such provision shall be reformed to the minimum extent necessary to
|
||||
make such provision valid and enforceable.
|
||||
No term or provision of this License shall be deemed waived and no breach
|
||||
consented to unless such waiver or consent shall be in writing and signed by
|
||||
the party to be charged with such waiver or consent.
|
||||
This License constitutes the entire agreement between the parties with respect
|
||||
to the Work licensed here. There are no understandings, agreements or
|
||||
representations with respect to the Work not specified here. Licensor shall not
|
||||
be bound by any additional provisions that may appear in any communication from
|
||||
You. This License may not be modified without the mutual written agreement of
|
||||
the Licensor and You.
|
||||
The rights granted under, and the subject matter referenced, in this License
|
||||
were drafted utilizing the terminology of the Berne Convention for the
|
||||
Protection of Literary and Artistic Works (as amended on September 28, 1979),
|
||||
the Rome Convention of 1961, the WIPO Copyright Treaty of 1996, the WIPO
|
||||
Performances and Phonograms Treaty of 1996 and the Universal Copyright
|
||||
Convention (as revised on July 24, 1971). These rights and subject matter take
|
||||
effect in the relevant jurisdiction in which the License terms are sought to be
|
||||
enforced according to the corresponding provisions of the implementation of
|
||||
those treaty provisions in the applicable national law. If the standard suite
|
||||
of rights granted under applicable copyright law includes additional rights not
|
||||
granted under this License, such additional rights are deemed to be included in
|
||||
the License; this License is not intended to restrict the license of any rights
|
||||
under applicable law.
|
||||
805
Deep_Art/deepdream/examples/00-classification.ipynb
Normal file
805
Deep_Art/deepdream/examples/00-classification.ipynb
Normal file
File diff suppressed because one or more lines are too long
5196
Deep_Art/deepdream/examples/01-learning-lenet.ipynb
Normal file
5196
Deep_Art/deepdream/examples/01-learning-lenet.ipynb
Normal file
File diff suppressed because it is too large
Load diff
5771
Deep_Art/deepdream/examples/02-brewing-logreg.ipynb
Normal file
5771
Deep_Art/deepdream/examples/02-brewing-logreg.ipynb
Normal file
File diff suppressed because it is too large
Load diff
947
Deep_Art/deepdream/examples/03-fine-tuning.ipynb
Normal file
947
Deep_Art/deepdream/examples/03-fine-tuning.ipynb
Normal file
|
|
@ -0,0 +1,947 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Fine-tuning a Pretrained Network for Style Recognition\n",
|
||||
"\n",
|
||||
"In this example, we'll explore a common approach that is particularly useful in real-world applications: take a pre-trained Caffe network and fine-tune the parameters on your custom data.\n",
|
||||
"\n",
|
||||
"The upside of such approach is that, since pre-trained networks are learned on a large set of images, the intermediate layers capture the \"semantics\" of the general visual appearance. Think of it as a very powerful feature that you can treat as a black box. On top of that, only a few layers will be needed to obtain a very good performance of the data."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"First, we will need to prepare the data. This involves the following parts:\n",
|
||||
"(1) Get the ImageNet ilsvrc pretrained model with the provided shell scripts.\n",
|
||||
"(2) Download a subset of the overall Flickr style dataset for this demo.\n",
|
||||
"(3) Compile the downloaded Flickr dataset into a database that Caffe can then consume."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"os.chdir('..')\n",
|
||||
"import sys\n",
|
||||
"sys.path.insert(0, './python')\n",
|
||||
"\n",
|
||||
"import caffe\n",
|
||||
"import numpy as np\n",
|
||||
"from pylab import *\n",
|
||||
"%matplotlib inline"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# This downloads the ilsvrc auxiliary data (mean file, etc),\n",
|
||||
"# and a subset of 2000 images for the style recognition task.\n",
|
||||
"!data/ilsvrc12/get_ilsvrc_aux.sh\n",
|
||||
"!scripts/download_model_binary.py models/bvlc_reference_caffenet\n",
|
||||
"!python examples/finetune_flickr_style/assemble_data.py \\\n",
|
||||
" --workers=-1 --images=2000 --seed=1701 --label=5"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's show what is the difference between the fine-tuning network and the original caffe model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"1c1\r\n",
|
||||
"< name: \"CaffeNet\"\r\n",
|
||||
"---\r\n",
|
||||
"> name: \"FlickrStyleCaffeNet\"\r\n",
|
||||
"4c4\r\n",
|
||||
"< type: \"Data\"\r\n",
|
||||
"---\r\n",
|
||||
"> type: \"ImageData\"\r\n",
|
||||
"15,26c15,19\r\n",
|
||||
"< # mean pixel / channel-wise mean instead of mean image\r\n",
|
||||
"< # transform_param {\r\n",
|
||||
"< # crop_size: 227\r\n",
|
||||
"< # mean_value: 104\r\n",
|
||||
"< # mean_value: 117\r\n",
|
||||
"< # mean_value: 123\r\n",
|
||||
"< # mirror: true\r\n",
|
||||
"< # }\r\n",
|
||||
"< data_param {\r\n",
|
||||
"< source: \"examples/imagenet/ilsvrc12_train_lmdb\"\r\n",
|
||||
"< batch_size: 256\r\n",
|
||||
"< backend: LMDB\r\n",
|
||||
"---\r\n",
|
||||
"> image_data_param {\r\n",
|
||||
"> source: \"data/flickr_style/train.txt\"\r\n",
|
||||
"> batch_size: 50\r\n",
|
||||
"> new_height: 256\r\n",
|
||||
"> new_width: 256\r\n",
|
||||
"31c24\r\n",
|
||||
"< type: \"Data\"\r\n",
|
||||
"---\r\n",
|
||||
"> type: \"ImageData\"\r\n",
|
||||
"42,51c35,36\r\n",
|
||||
"< # mean pixel / channel-wise mean instead of mean image\r\n",
|
||||
"< # transform_param {\r\n",
|
||||
"< # crop_size: 227\r\n",
|
||||
"< # mean_value: 104\r\n",
|
||||
"< # mean_value: 117\r\n",
|
||||
"< # mean_value: 123\r\n",
|
||||
"< # mirror: true\r\n",
|
||||
"< # }\r\n",
|
||||
"< data_param {\r\n",
|
||||
"< source: \"examples/imagenet/ilsvrc12_val_lmdb\"\r\n",
|
||||
"---\r\n",
|
||||
"> image_data_param {\r\n",
|
||||
"> source: \"data/flickr_style/test.txt\"\r\n",
|
||||
"53c38,39\r\n",
|
||||
"< backend: LMDB\r\n",
|
||||
"---\r\n",
|
||||
"> new_height: 256\r\n",
|
||||
"> new_width: 256\r\n",
|
||||
"323a310\r\n",
|
||||
"> # Note that lr_mult can be set to 0 to disable any fine-tuning of this, and any other, layer\r\n",
|
||||
"360c347\r\n",
|
||||
"< name: \"fc8\"\r\n",
|
||||
"---\r\n",
|
||||
"> name: \"fc8_flickr\"\r\n",
|
||||
"363c350,351\r\n",
|
||||
"< top: \"fc8\"\r\n",
|
||||
"---\r\n",
|
||||
"> top: \"fc8_flickr\"\r\n",
|
||||
"> # lr_mult is set to higher than for other layers, because this layer is starting from random while the others are already trained\r\n",
|
||||
"365c353\r\n",
|
||||
"< lr_mult: 1\r\n",
|
||||
"---\r\n",
|
||||
"> lr_mult: 10\r\n",
|
||||
"369c357\r\n",
|
||||
"< lr_mult: 2\r\n",
|
||||
"---\r\n",
|
||||
"> lr_mult: 20\r\n",
|
||||
"373c361\r\n",
|
||||
"< num_output: 1000\r\n",
|
||||
"---\r\n",
|
||||
"> num_output: 20\r\n",
|
||||
"384a373,379\r\n",
|
||||
"> name: \"loss\"\r\n",
|
||||
"> type: \"SoftmaxWithLoss\"\r\n",
|
||||
"> bottom: \"fc8_flickr\"\r\n",
|
||||
"> bottom: \"label\"\r\n",
|
||||
"> top: \"loss\"\r\n",
|
||||
"> }\r\n",
|
||||
"> layer {\r\n",
|
||||
"387c382\r\n",
|
||||
"< bottom: \"fc8\"\r\n",
|
||||
"---\r\n",
|
||||
"> bottom: \"fc8_flickr\"\r\n",
|
||||
"393,399d387\r\n",
|
||||
"< }\r\n",
|
||||
"< layer {\r\n",
|
||||
"< name: \"loss\"\r\n",
|
||||
"< type: \"SoftmaxWithLoss\"\r\n",
|
||||
"< bottom: \"fc8\"\r\n",
|
||||
"< bottom: \"label\"\r\n",
|
||||
"< top: \"loss\"\r\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!diff models/bvlc_reference_caffenet/train_val.prototxt models/finetune_flickr_style/train_val.prototxt"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"For your record, if you want to train the network in pure C++ tools, here is the command:\n",
|
||||
"\n",
|
||||
"<code>\n",
|
||||
"build/tools/caffe train \\\n",
|
||||
" -solver models/finetune_flickr_style/solver.prototxt \\\n",
|
||||
" -weights models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel \\\n",
|
||||
" -gpu 0\n",
|
||||
"</code>\n",
|
||||
"\n",
|
||||
"However, we will train using Python in this example."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"iter 0, finetune_loss=3.360094, scratch_loss=3.136188\n",
|
||||
"iter 10, finetune_loss=2.672608, scratch_loss=9.736364\n",
|
||||
"iter 20, finetune_loss=2.071996, scratch_loss=2.250404\n",
|
||||
"iter 30, finetune_loss=1.758295, scratch_loss=2.049553\n",
|
||||
"iter 40, finetune_loss=1.533391, scratch_loss=1.941318\n",
|
||||
"iter 50, finetune_loss=1.561658, scratch_loss=1.839706\n",
|
||||
"iter 60, finetune_loss=1.461696, scratch_loss=1.880035\n",
|
||||
"iter 70, finetune_loss=1.267941, scratch_loss=1.719161\n",
|
||||
"iter 80, finetune_loss=1.192778, scratch_loss=1.627453\n",
|
||||
"iter 90, finetune_loss=1.541176, scratch_loss=1.822061\n",
|
||||
"iter 100, finetune_loss=1.029039, scratch_loss=1.654087\n",
|
||||
"iter 110, finetune_loss=1.138547, scratch_loss=1.735837\n",
|
||||
"iter 120, finetune_loss=0.917412, scratch_loss=1.851918\n",
|
||||
"iter 130, finetune_loss=0.971519, scratch_loss=1.801927\n",
|
||||
"iter 140, finetune_loss=0.868252, scratch_loss=1.745545\n",
|
||||
"iter 150, finetune_loss=0.790020, scratch_loss=1.844925\n",
|
||||
"iter 160, finetune_loss=1.092668, scratch_loss=1.695591\n",
|
||||
"iter 170, finetune_loss=1.055344, scratch_loss=1.661715\n",
|
||||
"iter 180, finetune_loss=0.969769, scratch_loss=1.823639\n",
|
||||
"iter 190, finetune_loss=0.780566, scratch_loss=1.820862\n",
|
||||
"done\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"niter = 200\n",
|
||||
"# losses will also be stored in the log\n",
|
||||
"train_loss = np.zeros(niter)\n",
|
||||
"scratch_train_loss = np.zeros(niter)\n",
|
||||
"\n",
|
||||
"caffe.set_device(0)\n",
|
||||
"caffe.set_mode_gpu()\n",
|
||||
"# We create a solver that fine-tunes from a previously trained network.\n",
|
||||
"solver = caffe.SGDSolver('models/finetune_flickr_style/solver.prototxt')\n",
|
||||
"solver.net.copy_from('models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel')\n",
|
||||
"# For reference, we also create a solver that does no finetuning.\n",
|
||||
"scratch_solver = caffe.SGDSolver('models/finetune_flickr_style/solver.prototxt')\n",
|
||||
"\n",
|
||||
"# We run the solver for niter times, and record the training loss.\n",
|
||||
"for it in range(niter):\n",
|
||||
" solver.step(1) # SGD by Caffe\n",
|
||||
" scratch_solver.step(1)\n",
|
||||
" # store the train loss\n",
|
||||
" train_loss[it] = solver.net.blobs['loss'].data\n",
|
||||
" scratch_train_loss[it] = scratch_solver.net.blobs['loss'].data\n",
|
||||
" if it % 10 == 0:\n",
|
||||
" print 'iter %d, finetune_loss=%f, scratch_loss=%f' % (it, train_loss[it], scratch_train_loss[it])\n",
|
||||
"print 'done'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's look at the training loss produced by the two training procedures respectively."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"scrolled": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[<matplotlib.lines.Line2D at 0x7fbb36f0ad50>,\n",
|
||||
" <matplotlib.lines.Line2D at 0x7fbb36f0afd0>]"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": [
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAXUAAAEACAYAAABMEua6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
|
||||
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcXFWd9/HPtzt7AlkkJCGAgbCIqCSyuIDaRECEYZvB\n",
|
||||
"EQRFB5iMo8CjzuMwOlpdioo4IM4iM6wTgdHhgRFBRAhLM6gQtgQCIQQkYc8CJIEQQpb+PX+c01hp\n",
|
||||
"eqmqrl5SfN+vV7266tZdzr11+3tPnXvuLUUEZmZWHxr6uwBmZlY7DnUzszriUDczqyMOdTOzOuJQ\n",
|
||||
"NzOrIw51M7M6UlaoS2qUNFfS9fn1OEmzJS2SdLOkMb1bTDMzK0e5NfUzgAVAW6f2M4HZEbEbcGt+\n",
|
||||
"bWZm/azbUJe0PXAYcDGgPPhIYFZ+Pgs4uldKZ2ZmFSmnpv5j4P8CrSXDJkTEsvx8GTCh1gUzM7PK\n",
|
||||
"dRnqkv4MWB4Rc/lTLX0zke4z4HsNmJkNAIO6ef/DwJGSDgOGAVtLuhxYJmliRCyVNAlY3tHEkhz2\n",
|
||||
"ZmZViIgOK9LdUbk39JL0MeDvIuIISecAL0XEDyWdCYyJiLecLJUU1RbMNiepOSKa+7sc9cLbs7a8\n",
|
||||
"PWurJ9lZaT/1tiPA2cDBkhYBM/JrMzPrZ901v7wpIu4A7sjPXwYO6q1CmZlZdXxF6Zajpb8LUGda\n",
|
||||
"+rsAdaalvwtgSdlt6lXN3G3qZmYV68s2dTMzG8Ac6mZmdcShbmZWRxzqZmZ1xKFuZlZHHOpmZnXE\n",
|
||||
"oW5mVkcc6mZmdcShbmZWRxzqZmZ1xKFuZlZHHOpmZnXEoW5mVkcc6mZmdaTPQ11FSUUd1tfLNTN7\n",
|
||||
"O+iPmvo44HoV5fusm5nVWH+FegMwqh+WbWZW17oNdUnDJM2RNE/SAkk/yMObJT0raW5+HFrmMse2\n",
|
||||
"+2tmZjXS7Q9PR8Q6SQdGxFpJg4DfSToACOC8iDivwmW2hfkY4OkKpzUzsy6U1fwSEWvz0yFAI7Ay\n",
|
||||
"v66mXXxc/jumimnNzKwLZYW6pAZJ84BlwO0R8Uh+6zRJD0q6RFK5Ie3mFzOzXlJuTb01IqYB2wMf\n",
|
||||
"ldQEXADsBEwDXgDOLXOZpc0vZmZWQ922qZeKiNWSbgD2iYiWtuGSLgau72gaSc0lL1toZhypPd41\n",
|
||||
"dTMzIFeUm2oxr25DXdI2wMaIWCVpOHAwUJQ0MSKW5tGOAeZ3NH1ENG82v6I+BzyHa+pmZgDkSnJL\n",
|
||||
"22tJhWrnVU5NfRIwS1IDqbnm8oi4VdLPJE0j1boXAzPLXObYPL5r6mZmNVZOl8b5wPs7GP65Kpc5\n",
|
||||
"DngS19TNzGquP64oHUsKddfUzcxqrL9uE+CauplZL+jPmrpD3cysxvo01FXUUGAwqfeLm1/MzGqs\n",
|
||||
"r2vqY4FVpNsMuKZuZlZj/RHqLwOvAsNV1OA+Xr6ZWV3rj1BfGYUIYDUwuo+Xb2ZW1/o61Mfxpzs8\n",
|
||||
"rsTt6mZmNdVfzS+Q2tbdrm5mVkP90vySn6/CNXUzs5rq7+YX19TNzGrIzS9mZnWkP5tffKLUzKzG\n",
|
||||
"+rtN3TV1M7Ma6utQHw68np+7pm5mVmN9HeqDgI35+avA1n28fDOzutbXoT4Y2JCfvw4M6+Plm5nV\n",
|
||||
"tf4O9eF9vHwzs7rmUDczqyNdhrqkYZLmSJonaYGkH+Th4yTNlrRI0s2Syu3FUhrq63Com5nVVJeh\n",
|
||||
"HhHrgAMjYhrwPuBASQcAZwKzI2I34Nb8uhylJ0rdpm5mVmPdNr9ExNr8dAjQSOqKeCQwKw+fBRxd\n",
|
||||
"5vLc/GJm1ou6DXVJDZLmAcuA2yPiEWBCRCzLoywDJpS5PIe6mVkvGtTdCBHRCkyTNBq4SdKB7d4P\n",
|
||||
"SdHZ9JKa33zxWUYy1aFuZlZKUhPQVIt5dRvqbSJitaQbgL2BZZImRsRSSZOA5V1M19z2XEV9GdfU\n",
|
||||
"zcw2ExEtQEvba0mFaufVXe+Xbdp6tkgaDhwMzAWuA07Ko50EXFvm8kpPlK4j/U6pKi20mZl1rLs2\n",
|
||||
"9UnAbblNfQ5wfUTcCpwNHCxpETAjvy7Hm23qUYiNwKY8zMzMaqDL5peImA+8v4PhLwMHVbG80hOl\n",
|
||||
"8KcmmPVVzMvMzNrpsytKczNLZ6FuZmY10Je3CWgEWqMQrSXDHOpmZjXUl6FeepK0jUPdzKyG+jLU\n",
|
||||
"2ze9QOoB41sFmJnVSH+HumvqZmY15FA3M6sjDnUzszriE6VmZnVkINTUfaLUzKxG+jvU/etHZmY1\n",
|
||||
"1N+h7uYXM7Ma6us2dYe6mVkv6uuauk+Umpn1ooHQ/OITpWZmNTIQQt01dTOzGunvUHfvFzOzGvLF\n",
|
||||
"R2ZmdaS/a+oOdTOzGnKom5nVkW5DXdIOkm6X9IikhyWdnoc3S3pW0tz8OLSbWbn3i5lZL+vyh6ez\n",
|
||||
"DcBXImKepFHA/ZJmAwGcFxHnlbks19TNzHpZt6EeEUuBpfn5GkmPApPz26pwWe1PlLr3i5lZDVXU\n",
|
||||
"pi5pCjAduDsPOk3Sg5IukTSmm8ldUzcz62XlNL8AkJtergbOyDX2C4Dv5Le/C5wLnNzBdM0A7MZ+\n",
|
||||
"7Mkb7d52qJvZ256kJqCpFvMqK9QlDQauAa6IiGsBImJ5yfsXA9d3NG1ENAOoqK8CO7R72ydKzext\n",
|
||||
"LyJagJa215IK1c6rnN4vAi4BFkTE+SXDJ5WMdgwwv5tZufnFzKyXlVNT3x84EXhI0tw87BvA8ZKm\n",
|
||||
"kXrBLAZmlrGsDk+UqihFIaL8YpuZWUfK6f3yOzqu0d9Y4bLeUlOPQmxUUa35vfUVzs/MzNrp7ytK\n",
|
||||
"wU0wZmY1M1BC3SdLzcxqYKCEumvqZmY10N+33gWHuplZzbimbmZWRwZCqPv+L2ZmNTIQQt0nSs3M\n",
|
||||
"amSghPqIPiyHmVndGggnSl/DoW5mVhMDoab+GjCyD8thZla3HOpmZnXEoW5mVkcc6mZmdWSgnCh1\n",
|
||||
"qJuZ1YBr6mZmdcShbmZWRwZKqI/qw3KYmdWtgRLqrqmbmdWAT5SamdWRbkNd0g6Sbpf0iKSHJZ2e\n",
|
||||
"h4+TNFvSIkk3SxrTzaxcUzcz62Xl1NQ3AF+JiD2BDwJfkrQHcCYwOyJ2A27Nr7viUDcz62XdhnpE\n",
|
||||
"LI2Iefn5GuBRYDJwJDArjzYLOLqbWTnUzcx6WUVt6pKmANOBOcCEiFiW31oGTOhmcoe6mVkvG1Tu\n",
|
||||
"iJJGAdcAZ0TEq5LefC8iQlJ0Ml0zAB9lBA/zQQr8pt0orwEjVZSiEB3Ow8ysnklqAppqMq8oI0cl\n",
|
||||
"DQZ+DdwYEefnYQuBpohYKmkScHtEvKvddBERAlBR64CxUYjX3zL/ot4ARkch1vV4jczMtnCl2Vmp\n",
|
||||
"cnq/CLgEWNAW6Nl1wEn5+UnAtd3MqrPmF3ATjJlZTZTTpr4/cCJwoKS5+XEocDZwsKRFwIz8ukMq\n",
|
||||
"qiEva1MnozjUzcxqoNs29Yj4HZ2H/0FlLmcwsKGLNnOHuplZDfTVFaWdXU3axqFuZlYDfRXqXbWn\n",
|
||||
"g0PdzKwmHOpmZnXEoW5mVkf6sk29u1D3PdXNzHqoL2vqPlFqZtbL3PxiZlZHHOpmZnXEoW5mVkcG\n",
|
||||
"0olSh7qZWQ8NlBOla3Com5n1mJtfzMzqiEPdzKyOONTNzOqIT5SamdWRgXKi1KFuZlYDbn4xM6sj\n",
|
||||
"DnUzszoyUEJ9DTBKRVX169lmZpZ0G+qSLpW0TNL8kmHNkp5t90PUXeky1KMQG0i19THlFtzMzN6q\n",
|
||||
"nJr6ZUD70A7gvIiYnh+/7WYeQ4E3uhlnObBtGeUxM7NOdBvqEXEnsLKDtyppKikn1FcA4yuYp5mZ\n",
|
||||
"tdOTNvXTJD0o6RJJ3TWblFtTd6ibmfXAoCqnuwD4Tn7+XeBc4OSORpTUzJ4cSNCqZjVFREsn81xB\n",
|
||||
"bn5RUX8OzItCPFll+czMthiSmoCmWsyrqlCPiOUlhbkYuL6LcZtV1AjgpSh0GuiweU39NFJbvkPd\n",
|
||||
"zOperuy2tL2WVKh2XlU1v0iaVPLyGGB+Z+Nm5bapt50o3REYXU3ZzMzezrqtqUv6OfAxYBtJzwAF\n",
|
||||
"oEnSNFIvmMXAzG5mU26b+gdUVAOwAw51M7OKdRvqEXF8B4MvrXA5ldTUtyX1a9+6wmWYmb3t9dUV\n",
|
||||
"pZX0ftkxv3ZN3cysQgMp1Ntq6juQmnUc6mZmFRpIof4i8A7gnaReLw51M7MKDZhQj0KsJ93Y632k\n",
|
||||
"3jQOdTOzCvVlqK8vY7zlwN7AwzjUzcwqNmBq6tkK4N24pm5mVpWBFurLSWVyTd3MrAoDLdRXAJuA\n",
|
||||
"x4ERKqqxV0tlZlZnBlqoLweeyz+a8Sq+AMnMrCJ9FepDKL+m/kx+vho3wZiZVaTaW+9Wqtya+sPA\n",
|
||||
"dvm5Q93MrEIDKtSjELcBt+WXDnUzswoNtDb1Ug51M7MK9XqoqyiR2tTLufiolEPdzKxCfVFTHwJs\n",
|
||||
"iEK0VjidQ93MrEJ9EerVNL2AQ93MrGIOdTOzOuJQNzOrIw51M7M60m2oS7pU0jJJ80uGjZM0W9Ii\n",
|
||||
"STdLGtPFLBzqZmZ9pJya+mXAoe2GnQnMjojdgFvz68441M3M+ki3oR4RdwIr2w0+EpiVn88Cju5i\n",
|
||||
"Fg51M7M+Um2b+oSIWJafLwMmdDGuQ93MrI/0+N4vERGSotMRLuIUJjNZzWoGWiKipcxZO9TN7G1B\n",
|
||||
"UhPQVIt5VRvqyyRNjIilkiaR7oPesVO5Chgbc6K5wmW8CoxUUQ1VXI1qZrbFyJXdlrbXkgrVzqva\n",
|
||||
"5pfrgJPy85OAa7sYt6rmlxzkrwMjKi6dmdnbVDldGn8O/AHYXdIzkr4AnA0cLGkRMCO/7sxQKr+Z\n",
|
||||
"V5vXgJFVTmtm9rbTbfNLRBzfyVsHlbmMak+UgkPdzKwiA/mKUkihPqqGZTEzq2sDPdTX4Jq6mVnZ\n",
|
||||
"Bnqou6ZuZlaBLSHUXVM3MyvTQA91N7+YmVWgr37Ozs0vZmZ9YKDX1N38YmZWgYEe6m5+MTOrwEAP\n",
|
||||
"dTe/mJlVYEsIddfUzczKNNBD3c0vZmYVGOih7uYXM7MKbAmh7pq6mVmZBnqou/nFzKwCAz3U3fxi\n",
|
||||
"ZlaBLSHUXVM3MyvTQA91N7+YmVVgoIe6m1/MzCrQ7c/ZdUXSEuAVYBOwISL262A0N7+YmfWRntbU\n",
|
||||
"A2iKiOmdBDr07Ien1wNSUUOqnN7M7G2lFs0v6ub9qmvqUYjAtXUzs7LVoqZ+i6T7JJ3ayTg9aX4B\n",
|
||||
"h7qZWdl61KYO7B8RL0gaD8yWtDAi7uxgGdU2v4BD3cysbD0K9Yh4If9dIemXwH7A5qF+O5u4g4Ka\n",
|
||||
"BdASES0VLmYN7gFjZnVMUhPQVIt5VR3qkkYAjRHxqqSRwCFA8S0jHshr0RLNVZfQNXUzq3O5stvS\n",
|
||||
"9lpSodp59aSmPgH4paS2+VwZETd3MN6iHiwDHOpmZmWrOtQjYjEwrYxRH6h2GZmbX8zMytQXV5T2\n",
|
||||
"NNRdUzczK5ND3cysjvRFqM/v4fRufjEzK1Ovh3oUYl0PZ+GauplZmfqipt5TvlOjmVmZtoRQ9z3V\n",
|
||||
"zczKtCWE+oBsflFRE1TUNSrqnf1dFjOzNj2990u3JBRB9GAWC4FzVNRuwMvA/yHdv/3F/JgETAa+\n",
|
||||
"G4VY1UkZPgCsiODJkmHvAv4S2Bk4I4LV5RZIRe0K3AS0AscDZ1exXmZmNdcXNfV3VTORxGCJM2mO\n",
|
||||
"B4F/BG4BHgS2YcUeE3lyxgnAUcBOpFC/RkXtrKJuU1FfLJmPoPViaP2HkmHbALOBscAuwKfKLldR\n",
|
||||
"Q4FrgPOAmcBfVLN+Zma9QRE9qUR3M3MpIL4UwU8rn5avAz8AvhnB2SrqBOC5KESLxC+BI4G9I5in\n",
|
||||
"ohqBXwKH0Np4Adp0ImKPKMSLmvjggRwx8yZGLW1k62fORq1bsfCoo3j4+Dv51HGfpTmOJtXUm8oq\n",
|
||||
"V1HfB/YA/hxoBF4A9olCPFXpOpqZdURSRER3v1XR8bR9EOpXR6SasMT2wPKIjm/FK3EE8FfAfwBX\n",
|
||||
"AMcBvwD2iWBJHmcn4F7gR0BTBJ8EUFHDeeITe3HFby/kswe/xtRbHgLOZ+let7B+5HPc+v3RHPbl\n",
|
||||
"+3lptwZe3GN/PvK99YhNbBr8W/51wYms3GVaBE93uT5FTQduBPaKQizLwy4BlgA7knrpfC0K8Xx+\n",
|
||||
"7wxgqyjEWdVuQzN7+xnoob4a+DkwGjiaFMhHR7By83HZClgAXA18HvinCL4n8Q3go8BhEbRK/BPp\n",
|
||||
"xzm+mcf/TgQ/k5gA3ADMZ8SKI/m7iauBVh44ZXvu+urOvLT7ycBU4FDgcJo1F9gLOJb1I7/Gsvc+\n",
|
||||
"wg53XwT8KgrplsIAKmpYXt564A7g8ijERSXvHw78GrgMeI7UJPMDYAXwfdIvQ50ShbipNlu1exKT\n",
|
||||
"gfdFcGNfLdPMamegh/puwDF50H8AzcAnga+Sgu8HpJqugMYIPi8xHFgXQUgMJt2j/RekNvWrSc0u\n",
|
||||
"SySmA/8NPAW8H/jXPP//BJblYX+MYKbEnsDDwIURzNysnO+67iDGPnkDk+fALjdt5KVdj46L5sxW\n",
|
||||
"UVNIgT0Y+K+8HntHITa9OW1q+tk3CnF3fv1u4BzgY8ABpHb7/wL+Igpxlybf+3l2/N0pTJu1inc8\n",
|
||||
"di6D17UAXyEdYH4KPApsjEKs7XS7FrUDK/YYy3//zzHMnP4rBq9rJF25Owh4N7cXz2fYyg8y/dK7\n",
|
||||
"GPbKH/J2D+D1KMTSTub5UeCHwJejEPe3e0/AoCjEhpJ1PhB4D+kguLizspZLQnz0rP2Y8a1v5vX4\n",
|
||||
"HOl8x4nAT6IQj1c0v6JGAu8D5kQhWkvWY0oUYnFehwuApcD3oxDrVNRY4N9I+9k5+ecUa0JFDYlC\n",
|
||||
"rFdRDcAMYCvg91GI5R2MOwKYGIV4sv17ZS5rIrANqSK1FelOqYtruT4ly2ok7Vt7Ax8G7qFkm5eM\n",
|
||||
"N4r0zfuxKLzlh3TKXZaA7YAVUYie/PDOgKOitibly9woxEMDOtQ7KpjEMcBZwDtIJ0F3JNXiD4rg\n",
|
||||
"rTu52BmYQ+r1cmIEt5S8N4oUADdE8FQetiup18wlwBcj2JROmHI28KMIXuxgGQ3ABD7wk3+nqXgE\n",
|
||||
"oacZ/vIwFh1xAxuHPcee/28m8Gma43lSCG9PCtLrgDnte/ioqBFtwZzPB5xFawNsGPlOVr1zEU8c\n",
|
||||
"Opq9L9qaoasXIjYC19Da+EU2Dd4ONJhNg29h2CtXk/rpfwI4CPgNsBH4S9Zs20jj+tGodQnDXllD\n",
|
||||
"MBUQm4YuZuHRU2ltvJ0nPjGFo7/wWxpaTyR1Dd0qz2NJ3uZzgeeBg4HDgYuAU4EZUYhH80Htb4Bj\n",
|
||||
"Sbdavoj0jeVE0kHzQeAI4HHgKtLPFo7Pj02kwFwKvMwrk0ewZsJTbPfA3cA+wKlsGvRJkGjc8D+8\n",
|
||||
"MO0jbP3sNF7a7Up2/MNzwEkEwaoptzJ2yaHAY8CupIP/WmAI6acSh5K+sd1FOsexNelAegCwEniW\n",
|
||||
"tK8tBZoJDkdcCmwghf4KYF/g7vz3xly+52ltPJ+NwwYx5LVv5uX9nhRgytttEzAceIRUsdid9LsC\n",
|
||||
"B+Vt8iCpQrAP8CHgJWAdsCqX50N5fovzZ/IaMA7Yn9Sz6sK83n+fP/fHSQH9ev48JpCC+2Xgj8BD\n",
|
||||
"wGdI4fpCXs5rwHvzNJeSvs2uJR2UZ+Tt81iefi/SDzWszWV9iVQR+iNwAjA9l2E46YC7Sy6vSP9v\n",
|
||||
"vwc+SOqRdhepI8ZUkvGkytk+wM15uncABeAPwLtJ33IPyp/bE6QDxOWk/4Ef5XXbSOrivIB0X6nf\n",
|
||||
"ATdGIVbkA8cHgHeSKgb3k/bxrUn7/gZgWRQiVNSeuQx/yON+lLSv75rXcW0uf0Meb5f8ea8g9bp7\n",
|
||||
"mNREPBX4OCkHXif9psRo4Mn8ua4BJuay3JTX91DSPnMTcBvwJdL/VAtwVhTi3i0u1NN7NJBq5hvK\n",
|
||||
"mxf7AksjeKbM8d8FPFZNd0ptP+dAGjZcyIaR27J0+nxgMrTeDw3jSTvfBaSdZTqp9jGS9I81K4J/\n",
|
||||
"6qAsUxmzeCPjH/kui2dsig0jviAxmlEv3M0BP1zAwiNPZ8mMGcC5wH8xcukrvOcXn2e/nz7CS7vt\n",
|
||||
"z/N7r2T5e2Zx7HHraGgdyYX3LOT5fb8D/AvpH/NTDHl1HpuGrGHT0KdJtf1/JPXweR6YGcFaFTUa\n",
|
||||
"OJm0gz9Hql1tB/wvcGUUYqmK+hzBT9kwYh2D3mikYdOlpJ33ReA0UthcHoV4BEBFDSYF2dH5vRX5\n",
|
||||
"0UDamScRGseSj32YbR4bxqgX1iBeZ+24y5h12+dR61g++ON5PH3ANF7Z4Uye+OR3gD+jWQ1cOGcn\n",
|
||||
"nt/vcva5YAZ/9reNwELmf3oIzxywJ4ed9gDpILKRdAvoffPrV/LjrlzmU0ndTiexascWLrzvM5wx\n",
|
||||
"9U6Gvro98JEoxCoVtRewJ6kGODs3uf0dr044lQ2jtmfR4b9g2qwrGLZ6b9I/Nnm7iXSQ2wvYgRS4\n",
|
||||
"d5IODFPzPN8AFnDx75/g8zPWMeiNMcBDOVjE7c27snTad5l8z3p2v+4BJjy8hBRUyvvZGFJQrAZ2\n",
|
||||
"y49hpIPCsjx8LKmXWVtgXhSFePN3gXMNdzrw16TAH0lqBv0N6cCwOynMHsvTDyIF7vg83R6kHl8t\n",
|
||||
"pN5ma0mh+0T+rCn9FqCitiOF+yZSQLYCq/L+NQ44PW+rVuB7pN5rTwFXkr6Jj8xlaiJ1SFhJqtR9\n",
|
||||
"Nc+j7VvY3qSD0yHAiPxZ3J+XGXldd86fwWpSBWAT6UC2HbCcVLlpqxhckrfL1DysNY+/Oq9rI+kb\n",
|
||||
"0HjSgfcE0sF4NqkX3Bjg26QD3E552Vvlz+lgUvhfTPo2uDupZn4I6cD1gyjEije3YQ9CnYjotUea\n",
|
||||
"fe/Nv3fLHkMg3pWfj4T4GsQREEPbjSeInSD2hVgCcTzElyHuyc+/CvEixIr8mFAy7TshbodYBfEQ\n",
|
||||
"xLSSeV4BsQziaIhPQNwNcRnEcXk+H4QYnp//HOIqiFMgHoPYtqTcV0A8AXEfxG8gppYsfyTEdvn5\n",
|
||||
"cIgTIH6GNqxg/MP3MHTVcogP5GWdADEKohHi/XndvghxEMQOEFtD/Divx/g8z10gxkMUIG6BTQt4\n",
|
||||
"7xUzmX7RKIjfQpwPsSfEXRCfyNN8CmIBxGCI2RB/gLgmfx7/lrfVaogZefxdS7dpB5/joRDn5G36\n",
|
||||
"nxDPw6bzaKaxm89/XN7+n4K4KZej0+XkaQZD/FX+7K8oWac9IDZAHN7BNN/J8/9ZXobyY5v+/h8o\n",
|
||||
"8/+ky+3Y7fTNNNBMQxfvb08zh9CcKqCdjCOaaezoM6WZEaXT0swUmvk4zQzOr7ejmZHt1mkiRKdl\n",
|
||||
"6qIMnZYxjzO8/O1KVL1Ne/cDr75gW+IDYi+INTmkPp3/ue+CmJL/UYd0Mp0g1G7YIIhRJa9HQtwM\n",
|
||||
"8Vxb+Ofh5+RgH9/FvA+A2I90YHoRogXi3lzWl3IQr4C4EWImxJQ87eEQayEezQeEl/PjEYj/hriI\n",
|
||||
"dFB6AWJTDs1z8zp/Ly9rVQ7H7SAOgXga4kmIX0AM6qS8N0P8ewrgGJ3X+W6IayG2IR3kni5Zn1UQ\n",
|
||||
"d0L8EOLb+SBwIMTQvKxnIM7O6/revA5j8nb5YV63n0Ic2/bPDPHPED/Nzxsgirk8V5EOUvtCfAHi\n",
|
||||
"DojLIS6AWApxK8RhEH+Txz8K4pa8nW7J87sA4jyIYXnb7J6XMZd0EPkuxDqIY8rc746DOKHCfXUk\n",
|
||||
"xF4dDG/MZbsVYlweNikPuxri/XnYqLwPzMlln0SqUOxSMq/d87bZuQ//B3eFmNxu2BCIbSGGtxve\n",
|
||||
"kPfTr5EPohBTIVZCzOxk/g2U/K/mef8NFR4Eul8Pouppe3cDV1+wLfUBMZke1l66mPcgiK3bDdsa\n",
|
||||
"Yo8K5rFjDrwP53/sxvx8aifjD2/biUk1mEmdjDcs/xXELIhfQ0zI/wRDS8Y7k1zL7qKMe0JshPh2\n",
|
||||
"fn1cDtlBJeP8mHRw2SWH46EQ38rhfTrEcogLIa4r+Uf9Xp72CojHSYH/7Ry8p5O+zdwDcT/p2802\n",
|
||||
"7cq1F+kbyo9I3yZuytOeDPF1iF3bjb8v6VvFgxAjcsh/E+KPEIty2W4oGX8GxCsQC0kHrudJB86/\n",
|
||||
"Jn0bEsT0HEQPQPwLxDdIB60XyQfjTrZpI0QT6eD0q1yul/J6j4H4D1JI3wJxG8RP8vb9Vd5250Oc\n",
|
||||
"kcu0GOJZiItJB9vPkb5FPg5xbV7eyblMl0E8BfEZiOtJlZChpG81Y/K478jLPpN08O+yxtvJ+o3P\n",
|
||||
"+9wruUwNpArBhXk9l0O8Tvpm1LY//wjid6RvSS+TDqYPQlyZP9+2b02D8vgHQrwG0Zo/+6F5ewbE\n",
|
||||
"sbX9Xyeqnbbf2tTNuiJxJPC/EXR26wcBDRFvtnG3f/84Ui+o90WwSGIq8HwEr+cT74cAl0XwRsk0\n",
|
||||
"DaR2zleBWyJo7WDWla7Hh4GVETyau+eeReoZtYp0cvao2PzE/1eAqyN4RmLHXJ69gcNI5w9eI7U7\n",
|
||||
"30g6gf4B0rUdx5PaoBfm13eTTnBOIJ24nAo8A/yWdAL3VtJ1FXeQTgJfQzoPMQa4KIL1Ep8lte3/\n",
|
||||
"KvJtNHJvtB1JJx/nkdrOr+dP5xbuJp1v2gs4IoKFEqeQzuVcmtfjfaSTj5DOF3yGdNJzcF7fBtKJ\n",
|
||||
"xPv4Uxv5aNJndiypff3XpLb4e0lXm/8KeBr4el63y0jt8UuBb+XtuS3phOZrpPbyHYEPRfBy3tZn\n",
|
||||
"kT77L+d1+wZwUl7Hc4Bvkc6htfXGG53X5cy83Pfksm+VH/vk9yeROlXcBJwCbAucGsEGiUl5mw/K\n",
|
||||
"j2cieLHfTpRKOhQ4n3QC4eKI+GG79x3q1m8ktorg1f4uRxuJrYGPRXB9fj2ms4NWB9MOJvXq+GPE\n",
|
||||
"W0/+SwwhncicT+rltRcp9F4gBdtTETzXwXQ7ATtHcGtVK5Xm8RPg9xFcJXEYcAapl9qKDsYVqWdS\n",
|
||||
"W1BfQgrQ09rWS+IdpB47++THzqSD4L2kLsz7kw6Mz5JCdj7p5OWH88Ho/aTgv510fcvGkuWPIN0W\n",
|
||||
"5GXgruigJ1we7wukLsa3kTLuLODfIvhZfn846UB4GekgewfphOqHSMG+jnSgeoC0/T9C6vVyZV6f\n",
|
||||
"l/Pwk0knUjfmR3ME1/ZLqEtqJJ0tP4jUk+Je4PiIeLRkHId6jUhqioiW/i5HvfD2rK3+2p75osNz\n",
|
||||
"gO9FsKhk+BHAneUeNDuY7zBSL51/jqDbH/rJ18wcC1wcQZfXbeQDy/Wk3jendXRg6Ul29uQujfsB\n",
|
||||
"T0TEklz7beZeAAADf0lEQVSIX5BusPVoVxNZ1ZpINTGrjSa8PWupiX7YnhEsIzWRtB9+fQ/nu450\n",
|
||||
"sCh3/LmkZqdyxl1L6t7YK3pyl8bJsFmf8WfzMDMz6yc9CfXeO8NqZmZV6Unzy3Okq+ja7ECqrW8m\n",
|
||||
"3f/FakFSob/LUE+8PWvL23Ng6MmJ0kGkE6UfJ12Kfg/tTpSamVnfqrqmHhEbJX2Z1PeyEbjEgW5m\n",
|
||||
"1r969eIjMzPrW73yG6WSDpW0UNLjkv6+N5ZR7yQtkfSQpLmS7snDxkmaLWmRpJsljenvcg5Uki6V\n",
|
||||
"tEzS/JJhnW4/Sf+Q99eFkg7pn1IPTJ1sy2ZJz+b9c66kT5a8523ZBUk7SLpd0iOSHpZ0eh5em/2z\n",
|
||||
"lvcryLX+RtJtKqeQLvudB5R9bxI/3tyOi4Fx7YadA3w9P/974Oz+LudAfZCu4JsOzO9u+5Fupzwv\n",
|
||||
"769T8v5b0xs0bcmPTrZlAfhqB+N6W3a/PScC+Y6sjCKdm9yjVvtnb9TU37woKSI2kO6RcFQvLOft\n",
|
||||
"oP0VZUcCs/LzWaR7mFsHIuJO2PwnE+l8+x0F/DwiNkS6mO4J0n5sdLot4a37J3hbdisilkbEvPx8\n",
|
||||
"DemCzcnUaP/sjVD3RUm1EcAtku6TdGoeNiEi/eA16X4RE/qnaFuszrbfdmzeHdf7bHlOk/SgpEtK\n",
|
||||
"mgq8LSsgaQrpW9AcarR/9kao+8xrbewfEdNJv+f6JUkfKX0z0vcyb+sqlbH9vG27dgHp132mkW4a\n",
|
||||
"dm4X43pbdkDSKNJNwc6IiM1uPNeT/bM3Qr2si5KsaxHxQv67Avgl6evWMkkTASRNgrf+nqt1qbPt\n",
|
||||
"136f3T4Ps05ExPLISD/R1tYc4G1ZBkmDSYF+eURcmwfXZP/sjVC/D9hV0hRJQ4BPk+5hbGWSNELS\n",
|
||||
"Vvn5SNJ9pNt+5Lrt5kUnAdd2PAfrRGfb7zrgOElDJO1E+r3Oe/qhfFuMHDptjiHtn+Bt2S1JIt1y\n",
|
||||
"eEFEnF/yVk32z57cJqBD4YuSamEC8Mv02TMIuDIibpZ0H3CVpJNJP3j7l/1XxIFN0s9J99zeRtIz\n",
|
||||
"pB8EPpsOtl9ELJB0FenHhzcCf5troEaH27IANEmaRmoGWAzMBG/LMu0PnAg8JKntzo7/QI32T198\n",
|
||||
"ZGZWR3rl4iMzM+sfDnUzszriUDczqyMOdTOzOuJQNzOrIw51M7M64lA3M6sjDnUzszry/wFBsEB8\n",
|
||||
"UlvRigAAAABJRU5ErkJggg==\n"
|
||||
],
|
||||
"text/plain": [
|
||||
"<matplotlib.figure.Figure at 0x7fbb37f20990>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"plot(np.vstack([train_loss, scratch_train_loss]).T)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Notice how the fine-tuning procedure produces a more smooth loss function change, and ends up at a better loss. A closer look at small values, clipping to avoid showing too large loss during training:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[<matplotlib.lines.Line2D at 0x7fbb347a8310>,\n",
|
||||
" <matplotlib.lines.Line2D at 0x7fbb347a8590>]"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"image/png": [
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAXgAAAEACAYAAAC57G0KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
|
||||
"AAALEgAACxIB0t1+/AAAIABJREFUeJzsnXeYHNWVt98jgXIY5ZyQMNlIJJMMwhhssI0Dxsbr8Dms\n",
|
||||
"zTpne9e73qa9tnFYrzMYe53WOeyuFzA4YBAYTEYiCQQCCSRAaZQTEtL5/jj3TlXXVHdX9/SMZsR5\n",
|
||||
"n2ee6a6uqq5Ov3vu7557rqgqjuM4zv5Hv319AY7jOE734ALvOI6zn+IC7ziOs5/iAu84jrOf4gLv\n",
|
||||
"OI6zn+IC7ziOs59SSOBFpL+ILBSRK6s8/g0ReURE7hGRea29RMdxHKcZikbwHwQWA52S5kXkXGCO\n",
|
||||
"qh4MvAu4rHWX5ziO4zRLXYEXkanAucB/ApKzy3nAjwFU9TagTUQmtPIiHcdxnMYpEsF/Ffg4sLfK\n",
|
||||
"41OAFan7K4GpXbwux3Ecp4vUFHgReTmwRlUXkh+9d+yaue/1DxzHcfYxB9R5/GTgvOCzDwJGiMh/\n",
|
||||
"qepbUvs8CUxL3Z8atlUgIi76juM4TaCqtQLsqkjRYmMicjrwMVV9RWb7ucD7VPVcETkR+Jqqnphz\n",
|
||||
"vHIxy4EXaUmXNXyhZbkK+I6W9KpGj90fEZGLVfXifX0d+wP+XrYWfz9bi4hoswJfL4LPouEJLwJQ\n",
|
||||
"1ctV9WoROVdElgLbgLfVOH4QsLOZCwWeAQY2eazjOM5zjsICr6o3ADeE25dnHntfwdN0ReB3AQOa\n",
|
||||
"PNZxHOc5R0/PZPUIvnUs2NcXsB+xYF9fwH7Ggn19AY7R0wI/EBPqZvAIPoWqLtjX17C/4O9la/H3\n",
|
||||
"s/fQ0wL/rJa0Wj59PTyCdxzHaYCeFvhm7RnwCN5xHKch+prAewTvOI5TkL4k8M/gEbzjOE5helrg\n",
|
||||
"mx1gBY/gHcdxGsIjeMdxnP2UviTwPsjqOI7TAH1J4D1N0nEcpwH6mgfvEbzjOE5BPIJ3HMfZT+lL\n",
|
||||
"Au8RvOM4TgP0NYH3CN5xHKcgfUngPU3ScRynAfraIKtH8I7jOAXxCN5xHGc/pS8JvEfwjuM4DdCX\n",
|
||||
"BN4jeMdxnAboSwLvaZKO4zgN0JcGWX2ik+M4TgN4BO84jrOf0tcE3iN4x3GcgtQVeBEZJCK3icgi\n",
|
||||
"EVksIpfk7DNfRDaJyMLw9y9VTueDrI7jOD3EAfV2UNWdInKGqm4XkQOAm0TkVFW9KbPrDap6Xp3T\n",
|
||||
"+UQnx3GcHqKQRaOq28PNAUB/YH3OblLgVF2J4HcDB0hZetpWchzH6ZMUEksR6Scii4DVwPWqujiz\n",
|
||||
"iwIni8g9InK1iBxe5VRNC7yWVPGBVsdxnMLUtWgAVHUvMFdERgJ/FJH5qrogtcvdwLRg45wD/A54\n",
|
||||
"XqcTXcqb5GI5I9xbkDlHEaIP35WegOM4Tq9FROYD81tyLlVt9Mk/DexQ1X+vsc8y4FhVXZ/aplzM\n",
|
||||
"XC3pPU1fbFnWAYdpSdc2ew7HcZy+hIioqhaxwDtRJItmrIi0hduDgbOAhZl9JoiIhNsnYA1Hnk/f\n",
|
||||
"lUHWeLxbNI7jOAUoYtFMAn4sIv2wBuEnqvoXEbkIQFUvB14LvFtEngW2AxdWOVdXrRX34B3HcQpS\n",
|
||||
"JE3yPuCYnO2Xp25/G/h2gedrhcB7qqTjOE4B+tJMVnCLxnEcpzB9TeA9gnccxylITwv8ri4e7xG8\n",
|
||||
"4zhOQXpU4LWke7t4Co/gHcdxCtLXpv17BO84jlOQvibwHsE7juMUpK8JvEfwjuM4BelrAu8RvOM4\n",
|
||||
"TkH6osB7BO84jlOAvibwvvC24zhOQfqawHsE7ziOU5C+JvA+yOo4jlOQvibwPsjqOI5TkL4m8B7B\n",
|
||||
"O47jFKSvCbxH8I7jOAXpawLvEbzjOE5B+prAewTvOI5TkL4o8B7BO47jFKCvCbxPdHIcxylIXxN4\n",
|
||||
"j+Adx3EK0tcE3iN4x3GcgvQ1gfcI3nEcpyB9TeB7bZqklGWelGXWvr4Ox3GcSE2BF5FBInKbiCwS\n",
|
||||
"kcUickmV/b4hIo+IyD0iMq97LhXo3WmSHwReva8vwnEcJ1JT4FV1J3CGqs4Fng+cISKnpvcRkXOB\n",
|
||||
"Oap6MPAu4LLuulh6cQQPTKb3Nj6O4zwHqWvRqOr2cHMA0B9Yn9nlPODHYd/bgDYRmdDKi0zRmyP4\n",
|
||||
"SfTea3Mc5zlIXYEXkX4isghYDVyvqoszu0wBVqTurwSm5p+ry56/R/CO4zgFOaDeDqq6F5grIiOB\n",
|
||||
"P4rIfFVdkNlNsofln21gWWTXnnBnQc556tErI3gpy0BgNL3w2hzH6VuIyHxgfivOVVfgI6q6SUR+\n",
|
||||
"DxwHLEg99CQwLXV/atiWwzNfVGVrw1eZ0FvTJCeF/4P26VU4jtPnCYHvgnhfRErNnqteFs1YEWkL\n",
|
||||
"twcDZwELM7tdAbwl7HMisFFVV1c5ZVfFubdOdJoc/vfGa3Mc5zlKPU98EnBd8OBvA65U1b+IyEUi\n",
|
||||
"chGAql4NPCYiS4HLgffUOF9XBX4LMELKkrWE9jUxgneBdxyn11DTolHV+4BjcrZfnrn/voLPd2Dx\n",
|
||||
"S8u5npJul7LsAYZhYt9bmAysxQXecZxeRE/PZG2Ff74WGNeC87SSScBy3IN3HKcX4QLfGiYDy/AI\n",
|
||||
"3nGcXoQLfGuYhAu84zi9jJ4W+C558IF1wNgWnKcTIswXqUj5LIpH8I7j9Dr6dAQvZZkkZanwvaUs\n",
|
||||
"35OyHN/oSUUYBPwSeHET1+QevOM4vY4+LfCYIN8lZTkWQMpyJPD3wGHVDhZhugjn5Dz0dmACDYp0\n",
|
||||
"mMU6HJvc5RG84zi9hr5o0aQFfjbwI+APUpb5wMeAbcCYGse/EKt62YEIBwKfwHL9G43CJ2J1enbg\n",
|
||||
"Au84Ti+icKmCFtGyCD5EzuOArwJ3Ar/GXs93qe3RD6azEL8Yi8D/SuMCPxl4mt47y9ZxnOcofVbg\n",
|
||||
"genAk1rSZ4HrpSzvxCpbKnB0jePzBH4i8DCwk8YFfjTQjgu84zi9jL4s8DOxgU0AtKT/ByBleR21\n",
|
||||
"LZrBdBbxNmAjJvCjGryekcAmTOB9kNVxnF5DX/bgZ2KpiVnaqW3RDKJzpD0K2EBzEXwU+J0553Uc\n",
|
||||
"x9ln9MUsms3hPIeRiuBTrCNE8FKWfsGrT5Nn0YwiieCbEfiNwG6gv5Slry1k7jjOfkqfE3gtqWIi\n",
|
||||
"fjz5Ap+O4F9P5zVi8wS+jeYj+DZgU7iuXrkgieM4z036okUDZtMcQ3WLZkwoKXwwMD7zeK0IfgfN\n",
|
||||
"WzTQXAPhOI7TLfS5CD6wFhgCLBdhsAh3xge0pDuAZ4GhWKbNsMyxrY7g0wLvmTSO4/Qa+ozAizBF\n",
|
||||
"hP7h7jrM834aGAEck1nQO9o007FZpmnqefCDG7w0F3jHcXolfUbggZ9hs1DBIvjHtaR7sEheqBTm\n",
|
||||
"ONA6jc4Cn5dF01UPfmO47QLvOE6voS958KNIctTXkgywDgn/01ZMvQh+kAjpZf+6mkXjHrzjOL2O\n",
|
||||
"vjTRaUT4A1iCeeyQRO7DsJowYAL/POz15XnwEh7bLcLAcHs77sE7jrMf0Zcsmg6B15L+Wkv6j2F7\n",
|
||||
"XgS/DpiHlR8YkslNjw1CFOI2YKMqigu84zj7EX3Cogl2SjqCT1PNojkGs3F2kET7kC/wG8LthgRe\n",
|
||||
"ytI/PH9cANwF3nGcXkNfieCjjZL106F6BH8E8AQmvunjooBHIY7+OzQewY8AtmpJ9zZ5vOM4TrfR\n",
|
||||
"VwR+ROZ/mrQHH2nHGoQngK2kBX7ONSN5xTv30IIInkp7BjyCdxynF1FX4EVkmohcLyIPiMj9IvKB\n",
|
||||
"nH3mi8gmEVkY/v6lyum6Q+CrWTSQRPDJY2MeHswhVwr9n0lH8C7wjuPsdxTJotkNfFhVF4nIMOAu\n",
|
||||
"Efmzqj6Y2e8GVT2vzrmaTZNsVODXhf+dLZrB7QMYtrofx3x/JrznISrz2HcBB4rQX5U9Ba4rFhqL\n",
|
||||
"uMA7jtNrqBvBq+oqVV0Ubm8FHsRWMcoiOduyNBvBD8cW8mjEooGMwIvQnyHt9prnXHNM2Kcjgk9l\n",
|
||||
"0hQV6TYqI3j34B3H6TU05MGLyEws/fC2zEMKnCwi94jI1SJyeJVTdMWiWU31QdZnqRT4tVhe+9NU\n",
|
||||
"RvCDGbpmD+uet43RS+dKWQbz5rNexwE7NqeObaTgmFs0juP0WgoLfLBnfgt8METyae4Gpqnq0cA3\n",
|
||||
"gd/ln+X9zxORi8Pf/AaucwSwkuoWzVpS4q8l3QbMCsv5pQdZBzF09V6WnbmGYauPAt7F7GuP5vDf\n",
|
||||
"psscNBKFu8A7jtNSwphm1MmLu3KuQgIvIgcC/w38VFU7ibeqblHV7eH2NcCBIjK685m++aSqXhz+\n",
|
||||
"FjRwnbUEfjCwhsyMVS3pmnAzPcg6mKFr4eGXPc3AzbOAT7J+9kae9/t0SeFGCo65B+84TktR1QUp\n",
|
||||
"nby4K+cqkkUjwPeBxar6tSr7TAj7ISInAKKq63N27YoH/yTVI/hOAp+i0qIZ0t6P9udtYufIp4A7\n",
|
||||
"uPeNq5m4cFpq/0Yi+DwP3gXecZxeQZEI/hTgTcAZqTTIc0TkIhG5KOzzWuA+EVkEfA24sMq5uuLB\n",
|
||||
"r8EyXLKZOMUFvt+uwQza0I/NU7Zw99//BvgIj58Gw5+ck9q/qxaND7I6jtMrqJsmqao3UachUNVv\n",
|
||||
"A98u8HxdSZN8FFuPdTiQ7h0MAR4BplY5dgtWeAym3TKeZwft5dkhO7j2i0v0z198VD67fQADtk+S\n",
|
||||
"sgzSku7EPXjHcfYT+tJM1s3hL2vT5HrwKbZ2PDbuwYk8M3I3aSF+dkgbe/s/DMwN+zcq8O7BO47T\n",
|
||||
"K+lLAr8l/GUFvqpFI8LZ/OlLryJaNMOeGsfOthilD+woYiZ7b8MW8YauRfDuwTuO02voE9UkMYGu\n",
|
||||
"FsHHNMm8CP4INs6cRBT4Ie3j2DlyJ0mkPQR4hn57bwdOCMd0ZZDVPXjHcXoNfSmCb8aiGceOMQcS\n",
|
||||
"BX7QxrE8M3I7icDHnsF1wFmhbnyXPXgpyxgpy9sKnsNxHKdb6LUCL8JIEf4S7qYFPjubtSOCzyzD\n",
|
||||
"BzCeHW0DOo4ZuGU0O0duJRH44cAWLelj4RwvoDGBj9cViec9GagouCZlmS1l+UXB8zqO43SZXivw\n",
|
||||
"wATgRSIdC33U8uA3Y+UKsv73OHaOGkCM7gdsbWNn2xYyAh/2vQI4j4ICH6L9wcC21ObowU8HZkhZ\n",
|
||||
"0pbUHOC4eud1HMdpFb3Zg4+R+iHU9uAHY3VnkmyZhPHsbBvUca4Dt7axY/RmEq88nhcaFHhslajt\n",
|
||||
"qcU+SJ13OtAfSE+gGkOyaLjjOE6309MC30+E/gX3jUJ+GCamW8kIvAj9MEHdSb7Aj+OZEUOI67IO\n",
|
||||
"2DqcHaM3kUTasWcAcAcwmuk3DqKYwKej/0jsGcwI92enHhsDtElZilTddBzH6TI9LfC7KB7Fxwj+\n",
|
||||
"OGB7qM+e9eAHATtV2Uu1CF77DyWuyzpg23C2TthAjkUTIvE/ccRvplBc4LNF1+J5pwNLgINSj43B\n",
|
||||
"ovqhOI7j9AC9WeBHYIJ5HEmknPXgh2DiDZml+UQYhNk3Q9Eg/gO2DmXLlHbyPXiAexm3eDTNR/Bp\n",
|
||||
"D34BlRH82PDfbRrHcXqEnhb43YSBVhEGi1SdfQom5Aux+vPRJ8968NF/h84R/DgsffJZEBP/AVsH\n",
|
||||
"s372OioFPp0Fs5i25eMoVk2ymkUzHBgP/JXOETxY7rzjOE63sy8i+JhJ83HgX2vsOxy4C4umqwn8\n",
|
||||
"EGoL/FpgG3v7bwMuZPvY7WyZspHOefCRxQxbPYFiEfww8gV+ArbQyBI6e/DgAu84Tg+xLwX+aCqz\n",
|
||||
"TLKMAFYBK6gt8NGiqVxc26LoNcA29h64A/gX/vzFRWj/9EzWbBT+BP2fGcSwp/NWjspSLYK388Bj\n",
|
||||
"wOzUoOrY8Hpc4B3H6RH2pQd/BDCpxr7RPnmISg8+Lb71LBqL4J8duBP4H+5702asQYipkBUirSVV\n",
|
||||
"dratZPKd4wq8lrxB1p3h/xNa0vXAHpLIfQxWEdM9eMdxeoR94sGHAdA55C/eHYn2yRKas2iSCP7W\n",
|
||||
"D18GvBtrEHZQPYKHHaMfZ8J9Y6hPXgS/K/x/Ivx/lMSmGQMsxSN4x3F6iJ4W+I2YR30IZlcUieD/\n",
|
||||
"Ctwftm3G6r9EinnwN/zrai3pBjoLfLbUAGybsIwxS4qIcCeBD6mWu0kE/jHgICnLIMyaWoELvOM4\n",
|
||||
"PURPC/x1wFmYPXMLNvGpWibNCGCLKr9W5cth2yZgeJjgBIlgQy2BT3LP60fwG6c/wuileUsDZsmL\n",
|
||||
"4AnnjgL/CHAwFr23Axtwi8ZxnB6ipwX+GuBcTOAfwLJNqkXx2RRGwmSnrSQ2TTaCT/vziUVTKfC1\n",
|
||||
"Bllh5UlLGPXYkAKvZRidPXjC+aPAPwQcSiLwG/EI3nGcHqKnBf42bBLQWSQCX82Hz6YwRtIimRb4\n",
|
||||
"tZioR2IEv5VE4AdRL4Jf9NZlHLijv5RlVp3XUi2CLwMPh9su8I7j7DN6VOBVeRb4M7Z60v3UjuA7\n",
|
||||
"++PGRhKbI50muRyYmdqvWgQfs2jyPfjdQ3ew6P9tBj5a5+XkCryW9Fta0pgu+RA23jCOxKLpUYEX\n",
|
||||
"YYgIv+/J53Qcp3fQ0xE8mE2zG8soqWfR5EXIaZFMp0k+TlLkC+p78J3SJAM7uPkTO4G/k7JMqPE6\n",
|
||||
"ql1fB1rSLeF655FE8D3twY8BXpJTK99xnP2cfSHwVwJfVWU3VQQ+iFHeTFGobtGsBw4QoS0M3B6A\n",
|
||||
"Ree1BllR7ZicFNnJlikDgF8CH6jxOuoKfOAh4BRgHfvGohmKFTkrMq7gOM5+RI8LvCrtqnwy3H2K\n",
|
||||
"/Ah+KLAjDKpmyRV4VRSzaWZgJYaXhG3bgKEiHIgJ3a5QffJZ8gU6ToL6NXBajZeSN9Epjwex9V73\n",
|
||||
"iUVDIuxFMoMcx9mPqCvwIjJNRK4XkQdE5H4RyY1qReQbIvKIiNwjIvMKPn+1QdZqA6xQaXOk0yQh\n",
|
||||
"sWkOBxaHbTGCHwesCaIPFsVXE/jB7DlwJTAl/UBYa/XccLdaDyPLQ1hvoR3rUQwLq0H1FLH3MrLm\n",
|
||||
"Xo7j7HcUEZrdwIdV9QjgROC9InJYegcROReYo6oHA+8CLiv4/NU8+E4pkinSUXDaooFkoDWmYUIi\n",
|
||||
"8JOwyVWRZ/KeIwwEK7d8aB0wObNAx3zg31LXWETgHwz/28NEqK30rNi6wDvOc5S6Aq+qq1R1Ubi9\n",
|
||||
"FROsbNR9HvDjsM9tQJtIzQHKSDWBrxfBVxP4x6ku8BPpLPDVnuNPXPulz6FsJ6klAxbRz6yyHms1\n",
|
||||
"Hgr/23OuvyfoswIvZZkoZSnt6+twnL5KQ1aBiMzEMkJuyzw0BZuGH1kJTC1wyvXAEBGrvy7CiSKc\n",
|
||||
"S/UUSagUyKF0juCjRVNP4HdSXeDfAJzO9rE7qLRpJgOjsUYpux5rNZ4Or2VduN/TPnz04BsSeCnL\n",
|
||||
"BQXmAnQ3JwB/v6+eXMrSX8qSXcjdcfoMhQVeRIYBvwU+GCL5Trtk7munHUQuTv3ND354uibNa7Ef\n",
|
||||
"dC37I+3BZ22X5cCRWL2bx8K2tMA/ndq3agSvyibgvaw7ZCidBZ7wHEUGWK1CJbyIZPJTx/WLcIAI\n",
|
||||
"J6X3l7IMkrKcX+TcBbEIfuziSVKW2XX2TfMB4PQWXkczzAImSVmKruPbNFKWN0hZPp/Z/P+Ab3T3\n",
|
||||
"cztOGhGZn9bKrpzrgIJPeCDw38BPVfV3Obs8SWVt96lhWwWqenHOsSuwqPsx4HlY9F0rgk9HwFOp\n",
|
||||
"7Dk8Hs6xKJWBE2eyTsRqw0RyPfgUS9hw0CBm3JwV+J3AURTz3wHQkt6VupvugRwNXCXCuJDZA9ZD\n",
|
||||
"+qGU5X8L9hDqYQJ/3rsuBM4EXlnwuBkkywzuK2ZhmU/jgaelLKcBf9OSPtsNz3UwNgEvu62RRtFx\n",
|
||||
"uoyqLsCW/ARApHmbskgWjQDfBxar6teq7HYF8Jaw/4nARlVdXfAaHsBEHUycZ2OReU2LJuS6D8QE\n",
|
||||
"P7IWy6pZnNq2Dct4yRtkrSXSq9k8rR87R6SX3ZsM3I5F8IUFPu/6w+0RwGjec8TXpdyRnTQN68HM\n",
|
||||
"aeSkIpxVpXDbUAZsgcl3Hos1HvXPVZYDsJ5Lkbr43Um0iGIj+xvg2G56rrF0fs+nUcxqdJxeSRGL\n",
|
||||
"5hTgTcAZIrIw/J0jIheJyEUAqno18JiILAUuB97TwDU8ABwpwgHYAOkDwMnUH2SdCqxMpT2mc+Ef\n",
|
||||
"SO3fzCCrnWvH6DU8M+KQ1ObJwN9oMILPkK4oOQIUhj99IYn4xp7QMQ2e9wt0jkABhnD0j7ew+qgV\n",
|
||||
"wAgpSxHRnoJ9NwoLvJTl/G5I/5yF9QSnSlniWrcHt/g5ImOB6RnPfVp4bp8F3CBSlqG13jcpy1FS\n",
|
||||
"lrf24CX1KVr1nSuSRXOTqvZT1bmqOi/8XaOql6vq5an93qeqc1T1aFW9u4FruB/LepmJeeS3Y41K\n",
|
||||
"rQh+FJ3tmci9wB2p+00JPAA7Rq1A+80E+8JiPYZFWI+jkAefwzISkRrOhHt3MGjjWJI6OtOwAdlG\n",
|
||||
"I9U28hcLH8rxlx3Anf/wGHA3xRqOWPKhkMCHkg6/xT7HlhC+4LOw9QCmkCxg3l0CPwb7PaQHlqdj\n",
|
||||
"351uy0CSsgyXshzXXeffh/wBqDWW9BrgrT1zKX2SN0pZvtXVk+yLUgVZHsCE4RDMI78X+7FVE98t\n",
|
||||
"2EzTWVi2TgWqXKjKn1Obqg2y1sqiMXaMeYT+uyaGe5OxmbfLwvM3G8EvAuaG28M5/rLHefjc3WiF\n",
|
||||
"wF9B4xF8vsAfd9mhDNyyl0Vv3UpjAr+c4hH8yeF/Kwdlx2CzjR/ABH42sJcmBV7KcmCdXcZiFt+c\n",
|
||||
"sH8/7DN/jO61ac4HftcTA8k9ReglngK8vMZux1PZmDqVzCdJsW6a3iDwa7Af7mlYpsm9YXtuBB9s\n",
|
||||
"mE2YD54XwWf33x3OL1RG3TvCeaqzadr9DNgW7ZTJmF2wPNxvVuDvBY4MP+jhPO/Ksdzx3idBpgTv\n",
|
||||
"eyrwf8AxRbppUpZ+csSv+wFtjHx8lJTlnanHhjK/dDJ/+dyNaP8RmMAX6RnMAO6k+CDrKdiXsXUC\n",
|
||||
"f8uHXoayjCTldjaWnttsBH9LNlKWsrxIyhLHLcYCt5L48BOw3uKjdK/An4A1YC+qtoOUpaHxmCrn\n",
|
||||
"OEjKcp2U5dNdPVcBzsG+ay/Ns+3C9/o4YEqBhve5yhnA9V09yT4X+CDYDwCvxgT+vvBQLQHdiPng\n",
|
||||
"nSL4KmwDVqX9euDjmJBWZ+3h93DAzoHBl40RfBzIbUrgtaSbgNXAHGZeP4MBWwfy6FlXs2vYNkxI\n",
|
||||
"pmE/jq0Ui3Dezflv/Av9dvfjNW+6EPiulOX54bESTx+7gXvffDdmM9yFNRwTpSxfqnHO6WHfohH8\n",
|
||||
"KcAlwOkt86ufPvar7Bi9DmtUYwT/B+DgRp9DyjIaa9iOzDz0DZJ6Q2OxBiSK6XQsgCg6p6NZXgD8\n",
|
||||
"nCp2hZRlMHC/lGVG3uP1kLKIlOVd2Gu7H3hHD4wpvBy4FPudzs15fCoWcK3A3uemCPZWoUzAvoSU\n",
|
||||
"ZTqWaLG43r712OcCH7gfi8weVmUtyeSgajQl8OkNqjyuWmcm6p5BS9k2fi8m7pOBp0Je+3Kaj+Ah\n",
|
||||
"2jTP/+nRrJp7L3rAY2yduA3LIhqLvf67gJdIWU6p030/Htk7jwtfDZMWHg98C7goTFJ6B1ddvgxr\n",
|
||||
"mEZiFtg44FrgY1KWiVXOOQNraAfnTfSRspyYuj0YeD6WRrsNK/TWdUasGM7Wie0kAn8QNrayl8bT\n",
|
||||
"N08J/zui/yByM4Fp4TX0B+4hEfhp2MpcLRd4Kcs4KcuA8LyHAZ8EXiZlyfP6T8XGfpq1M34EXIR1\n",
|
||||
"+T+IBScvaPJcdQkR+VnA1Vhp8HNydjsO6yEuIxlbaYb/g47lPPcn5gMLgtZ0id4i8DHrJU4G+hxJ\n",
|
||||
"JJ/HBuxHXteiCWyj0n8vyuNsmtaPbWNnkkTwYALf7CArRIGfueAQHjn3VqCdjTOewbzs1VrSPVj3\n",
|
||||
"rMze/n9g15Az0wenbAWAw3j45Z9i+JNw88euBL6IzcT9D+AbbJo+IFz3yJBXfztwQzh/XnQFiQff\n",
|
||||
"TkZMpSxTMbsjTk47DlisJd0WztuUTSNlaZOynAcgwmDaHj+A9bO3UGnRPEpY5zYbhUpZBkpZpgbR\n",
|
||||
"zHIaVmIjbe+MxcZmpmJ+/7pw7rTAd1cE/zOsptFc4EEt6UpsIZyP5OwbP/uGI/gQGFwAnKElfSAI\n",
|
||||
"xi+x70dLCdH097AGZamWdBXVBf54rLFeRpMNV7DbDgH+n5Sl2qpwfZUzSOXBd4XeIvD3Y0XNHgdQ\n",
|
||||
"5duq1Mqj3xj+Nx3BF0GVZ9k+djsbDjqWSoG/jq51nxYB5zFk3XBuf+/dQDsbZu8BXkhotLSkX9eS\n",
|
||||
"jueO9+xl/ZzXZI6/WcrywiByh/G3jz/B5QvhhouXBbG4EWss/gMTsRjBg3Wf3wcsJCcvPpxzOvZZ\n",
|
||||
"rKWzTXN2+H90+H8KcHO43bTAY6m4Pw6iNJq2ZbBq7g4t6WZsVnS8priQ+delLHdJWc6SslyGjac8\n",
|
||||
"EF5zlhcCP6BS4GeG/1MxsV+HCc60EIVGi2YFKYGXsoyXshxNQaQsr5KypHsOw7DP5h3YD/n28NAH\n",
|
||||
"gbdJWV6fOcWZ2Oc5Mxz/DilLUYGejQUM6d7wL4DXdcOg7kex92kR8I9h243AYSEoSJOO4GcBSFk6\n",
|
||||
"9cqkLIfVsJM+DnwF+CHwT12++hyqjQ9IWQ6WsmTtvqbOL2W5OOezmE8L/HfoPQJ/N/CvVeq/57ER\n",
|
||||
"62quL7h/UwIPwMMvX8q4xR/CIrsnAbSk/64lvbqp8xn3AIez7EVr2T1sI9DOukP6Y9U6O3olIoxg\n",
|
||||
"2RkjGLKuI/NFyjIKs0ROw6yL7TxxavyCxOj1U8AbwopSQ7D3qZ8IA7WkO0Ikl87mSTMO2KEl3YoJ\n",
|
||||
"fPaHdzb2XsZjTwVuCrcX0LwP/xpsQtox9Ns1ijEPw8qT4/dhJWaPPYMJ/DnA3wHfw+Zd7MEmss0D\n",
|
||||
"zk/7siG99Sgsap6TurZZmA0YBb5dS7oL+4xnUt2ieTPWWBTlEuBjqfsvBm7B0j//kSDwWtKngFcA\n",
|
||||
"34oNQhg7OAT4FUkEfz5wacH5DEdgwVMHWtKHsc+1S6mZUpYxqdvjgfcD79WSfllL+pfwXDvDtb81\n",
|
||||
"jAUskLIsxoKCDoEPabYr0j1TKcsIrMG+POuzS1lmYg3f94AvYSmF1ezGZl/f84Hl4feW3t4fWyvi\n",
|
||||
"2no9BylX2NA3AAAgAElEQVTL66Qs36vxe5gDlLDxx3jMBCwY63IGDfQSgVdlmypfaOCQjWQmOdVh\n",
|
||||
"K80K/F0X/Y31c57GfhBPZR8WYZ4IVzW4JN4KYD33v34z5uWvY90hAzExTttOh7PyJBiy7vBUNsIL\n",
|
||||
"gF1YFHgY1pNIL2GIlnSxlvS6sG0o1sBtpnLRj2oCHyNlyETw4cv9YuDrwNxw/1QsUgOzdXZjYwmF\n",
|
||||
"CdHbsdiM6Rcz/zMnov1g6dnx830Ss2fABP71wNe1pN/Rkh6kJX2flnSDlvQx7P1LL9RyIrBQS/o0\n",
|
||||
"lhobq5zOwnoe00gieLBg4zyqWzSHYwPV6Qlw1V7XVKwRPl/KMiBsPhfzp7+GDaTFCB4t6T2YYMUZ\n",
|
||||
"4/PDNT5C0uM4HGscLsk810Apy6VSlqNSm4+kctJf5GbsfUkf32kWtJTljKzAhe2zgaekLPFz/jTw\n",
|
||||
"8/D+Z/k+1ls5H/uevhV4T7BwHsM+h1dgqcfp780xWCA0AxuwTXMScL2WdIuWdDXwP8Dbcp67K5yK\n",
|
||||
"fffLme3vwn5L3wJ+I2WptVLafGyG/yerPD4Hs5s/lWoEjgDub4X/Dr1E4JtgA8XtGbBlAm9t8rke\n",
|
||||
"5RdX3An8b5XnnA+8jNo5vxWED+9sFr82rirVzvqD4w8sLfBHsnUiPDtoJxbJgQn7z7Av+RGYt9yG\n",
|
||||
"NWJ5X7Yo8JuonLDzEDZLc5iU5Z9SXc6DqCLwmAivBq7CGoe5WGS9JvW6Otk0BSL68zAP+krgLA7+\n",
|
||||
"/Su55y1Av/jcaYG/DxtPqVY24zfABcGP/yU2+Pvr8NhSEptmJiaUaYsGrPfzT9j7vQILJg4Um0kL\n",
|
||||
"9p7fRTEf+8WYD70YGzAXTOB/jzWKbyFZLyDydayncTEmLtcQqqQGER6PFUE7V8ryAuh4f7+NBSHX\n",
|
||||
"SlliAbsjyUTwgVvJCDxwp5TlTfFOCCh+A/xfzkD7C7Be079LWV6KRaCfqfIe3I29h98HPqElvV1L\n",
|
||||
"+uPwWLRoXol9fw9NHXcs1jO8AHitlCVdE2o6SboyWC/undLamdQnYNH166UsH5KyfE7Kcin2Ot8P\n",
|
||||
"fB77DT0oZemo7yRlOTTV4zgMG+B+v1gdpSxzsN/yASRjFenFirpMXxX4dSQiVBdVLlXN/aIX4VE2\n",
|
||||
"T5umJX2NlnRHzuNzMcH7jEjx91NLehd6QKyauZFN04dhBccqBR620X7Ik9BRdfIkLGLZiP2wFmMz\n",
|
||||
"e58mM9Ep9CqGkCPwoWDXYqx65+eBd4eHXo6NMYC9z2mBPxv4I7AE+5G9jM6DQR0CH7rlXwDuk7LU\n",
|
||||
"KpH86vCabgSOZ+xDp3Pf320ksYduI/j8WtL7gIOC/ZTHb7GqpDdjP8BDtKSxImT078GE5V4sK2d2\n",
|
||||
"eK3RwvgqZhc9HRqtlVjOtmA/wE8DbwiNyAeyFkKKs7CMpZ9jYv56rBfxiJZUtaQ/yRaUCzbRe4GX\n",
|
||||
"AJ/FhDuOAxwBLNGSbsCiwm+HXtQnMdE9E4uWfxuuqZNFk3o/OzJppCzTsEj5c6lB6mPCe7Ia+Hna\n",
|
||||
"ksEaki+G9+KXwBu1pGvz3oDw/n0TuFFL+qfMw6ux93k+ZntlBf6uMH7wc+AfUo9Nx+yzyJ3Y7+HF\n",
|
||||
"edfQJCdgjes7sdf7DPZevk5Leq+WdK+W9B1YY/uT1HfgKqwRBxP4a4EPY2NG/UNDEXtfc7DEkq9i\n",
|
||||
"nxu4wAPwEyp9ze7kUUJFQRFOEOH7mcfnYa36XhqI4gMjgC2q7GHPwI1ov6eoFPgjgFtYeWI7cFL4\n",
|
||||
"MZ+ARWB/I8kOacPso2wEPwhbg3YPnSN4MJvmy5iQvDb41a/ARBI6R/CnAddpSXdjX8KLMEFPswDz\n",
|
||||
"4fsD38UGEm8FfpY3sBe2zQf+EER7IdsmrGX9wQ8RBF5LeqmW9EfxmODt5qIlXYo1FJ/Wkl4cuvCR\n",
|
||||
"dJbMTCyCXIF9hu2p/b6M/ZDjGMAKrGdjYx6Wj38A9oO/CPhKGDB7v5TloPC6BBOcP2Pv5yuwgcgP\n",
|
||||
"1et+a0mv1ZKepCX9lZZ0T3i96zEBjz/+n2LjUH/FBqhfFiyLq7Dg57zwWvO83IeBUcE7B2uQf4/1\n",
|
||||
"TN4ftp0dXudbMCFeImW5IDx2PBZdvwP4qJY0+x3Ivp4fhOvJbo8px7dh35FOAh9ufwuL0GNPokLg\n",
|
||||
"w3kux8S4y4ilq04HHtCSXqElfZOW9DPhe1gx+KklXYAFAM8P4wCzgROCtTUE633+BgvkfovV6XpJ\n",
|
||||
"OHwO1qv8C/DCVADx3BZ4VbarVvwgu5PHgJkhOj8dEyMARBiERYT3YStavazBc6fr3rdz5z+8l+RL\n",
|
||||
"DRbB38jD527FunDnY1kR6zCBh8SDf4rOpQrSC6LkCfxtWDZNCROxL5L41ZAaZA1fvmOxaAmscZhM\n",
|
||||
"Z4F/FJvE8mcsSj4T6x0MBr6TI/IHA2u4WDeLcCjwNW765N8wMa5M0SzYQ9KSnq8l/a+ch5ZiKZb9\n",
|
||||
"sIj1cezHOZfEokFLuktL+j+p427AovEjsJRQxayD52MDhi8J1/tJLLsDQkE6Leny8HmN1ZIeHwS4\n",
|
||||
"GZZjkeHicI2Kva8PA6doSdMR7fexVOPH8xrD0Gu4gySKPx1rmP8R+LhYCuxLgD+FQfn3YL2Pz4RI\n",
|
||||
"dR4WXd+gJc0GPLnUaNSWAr/DGqJDoUNgpxDsKy3pQ1hjGhuJGVRG8GDiebaUZVDcEBrdi6QsV0hZ\n",
|
||||
"XlXt2kJPc3Rq07HAohDIFOFmzDqNRRJfgEXvD4WemgIfwmyxM4FDw3XOwVJKn8Aa61guPW/cpCn6\n",
|
||||
"pMD3JGEy1EbsCzcXmCFCTJ86HFiqyk4skjq16HlFGIBNrok/wHau/vb62GUXYQwWAdzHo2eD/WB/\n",
|
||||
"gGVggH2p1mPRVbUIPvrvkC/wPwBOD8/5K8wa+FXq8XQEPx14JgyOgQn8ktR9oOOH/AcsIn6ZlnRr\n",
|
||||
"+KG8Eouar5SyfF3K8uFwyDyskTkZ+JWW9H+4892bMOHKZvDcIdL8zEdMhJ+HDbRuDrn7K7H3ZV2N\n",
|
||||
"467AxKVjKUgt6d1a0ie0pBuxxvcj2A/49WHg7b1Y5EbYvyvzJsAao5NIRXda0vu1pG9Vmx2d5leY\n",
|
||||
"pVNLKG4lEfj52MSah7HG4VLMokk33tcBA7DewlPhdbeCd2I9vYexxrc/9p24Vyvr/i8gyfzJWjRo\n",
|
||||
"SduxQCs9/vMWrBFcTu3o/mTgrtRY0fGkBr8L8DesoT8F+00dj2lDx/iKlvRuYEr4vyTsM4XEav4r\n",
|
||||
"ZlUeSLMJITm4wBcj2jQx6ySmrEVxAvtyTQ3CXIThmD0TI5t2Ktd/jWLSDv1GaUkvw6KCT0FHxsWx\n",
|
||||
"QVBzPXgqBT6bRUPwEeO4wq+xga505Jr24NNdZrBB54+Tzzu1pBeEtMb4XFswC+tG1s9+ht2DLgl5\n",
|
||||
"xsdg7+EYkmnro7Av/iAR0gN8E+hajfoHsPfknzF7BpKB81oCfy/2wzufnO6zlnSZlvR/1OYg3BrO\n",
|
||||
"/xpaO8tyOfZ7rdt9D43Jz0i+m3nchllp07H3O57337DP+vbQAMZzKvBfWPbOHbQILenq0GPahtWl\n",
|
||||
"mkHn7xrY7+uoEN0fQH6K9FVU2qRvAS7GxkxeGCzIPE7Cgo+YxXMCjQl8jOBPwXojG7CApmIAPTXe\n",
|
||||
"cif2XXoyjLmAWV7vJOkhtgQX+GI8itkls7DWOq7yMxeLZG1SlP24T8k7QQ7ZZQnXkS/wG7B1YNGS\n",
|
||||
"rtCSdqyUpSVdHm5WWDQivEqET5AMsEJ+BN+BlvRxYGJmsCwdwR+LZUTE/Z/Ukl5Z5Vy5K1FpSZ/R\n",
|
||||
"kn6Bbyy9kY2z9mIRZGwkR2MLuQzHBGd9znsyhGQR8YYJjdmrsZS65SKUWHtofH+qWn7hB3cl9iOu\n",
|
||||
"133+T6wR/rKWtOg8jSI8jqXHPlpvx8AHoGbq8Q3YBLI7gRviZxaE9g1k0jADP8Gqst6Z81greAgL\n",
|
||||
"Yl5O53Wf78Nsr+nAE1VE8Crg5cFymYVF0VeHHs4dwItD1tibxQrNxeDhBOzzf0mwTk4jsUCL8Aj2\n",
|
||||
"3ZyLNQy3Yb26bIZU5E7gdZg9FfkrNs7TMv8dXOCL8ijWIi/BPrROAh9oxKYZTmW9nWxZgIOxbut6\n",
|
||||
"kjVc+6fsoTRZi+Zw7Etaz4OvIB2xpa5pSJjQcQydo6pmaePRs3ZiPu88rOGIQj4NE/sNmMCn35Mh\n",
|
||||
"kLtqVWHUsnBeh2VmnMfy+XFMoFYEDzvargFg19B6P8CrsCJm3+zKdeawHLPECi1XGKLiqh5yiPJP\n",
|
||||
"xyyML2ceu1lLem3OMY9hPb3rso+1iIewAf9hVFqFYK+/DZtBnfXfIw9gmnYENiHtl6kIOdpsP8Je\n",
|
||||
"8zcx2xMs0PgqNrB8ATa+UO05OhEam79httJ2TOD7U1vgJ1Ep8A9iv3UX+H3Ao1g2yKJwe3YQ2qOp\n",
|
||||
"7AbfhE2LL0I2gs9aNDMxG6EjgsfSrfKqQGYHWUdi0UDaokmvJFWIICbfAf6FTATfRdp45GWKTXrZ\n",
|
||||
"FXz8+BqnURnBj4OOAdaBdFHgAbSkv9eSXgEMYdXcrdg4yPaaB/38qnu56RPw+a15tW7S596lJf2g\n",
|
||||
"5qfUdoW/0OIaMmEA8L+1pIXniGhJXx/swe7gQWyg9S3Zxin0MB7ABppzxTcI7fexaP2fMEspciWW\n",
|
||||
"0jgDK818AXBhGFAejn3PT8NKRmQnVhXhD1haJVgUvwtL0MjjvvB4h8CH1/dDkkmDLcEFvhiPYi1y\n",
|
||||
"h8Bjrf5S1Yo1YW8HjhIpVK2vnsDPApapsgNQEQZjX/6z0icRoT8memtIIviRWAMxjETgn6K5olmX\n",
|
||||
"ABdi3flOC6k3SRvLTx+ICXlsIEdjqaZR4LMRfBTWLgt8iqEsOW8T8JW6vueKUwZy7RehVdUyGyQ0\n",
|
||||
"HC3Lruil/BY4V0taLfK9D3gpNebAaEk/g/2O5mpJ70htfwyL0l+rJd2pJV2MBREfBu4Ig7SLsYDi\n",
|
||||
"941euJb0Mi1pXBz7duAd1XpboVexEHME0ts/piVtVS8ZsMEKpz7R91yEWR2zsXSnv6R3UmW7CO8H\n",
|
||||
"fiXCNaodk4fyyPPg03bETJLZeusxAZwFHCHCeFXWhMdGYlbPNmBwmNw0AhPEWSQCv4JkvdfCaEnX\n",
|
||||
"SVn+AzihhYM/bewZOBTlKqRjgZfR2Be+msDHxqtpDz6HoWyddKCW9F8K7Btnsh4GZCfsOC0gpJPW\n",
|
||||
"KrJ1HzYxr6Z9EmySJTnbs0kBv8IGYKNF9X1ANZn/0BSh9/HTOrudR41xn1bhAl+MdZivtgiLMg/C\n",
|
||||
"JrF8LrujKj8U4XZSKXJVqBrBi9CG9RjiIF20V2ZivvxpJJOR2oANqjwrwh4slS167UeSlDVeAUwT\n",
|
||||
"QRqo4RP5HPllEJrFZrWuOKXE9JtjSthorPbIYcAzquwSyRX41kbwxV9XWuCdfUMsIV7YH6/DrzDP\n",
|
||||
"/zYALen3WnTeumgo79HduEVTAFVUlRNV2aTKFiwqPh4bVM2jncRTrkYti2YmsDwlxOuxruNUzFec\n",
|
||||
"nzqujaR88g4sch+JWTJHUjnICplUySIEr7b24iiNYQL/g5tWq1VRhETgj4YO2ytdzbKlFk2wtmKB\n",
|
||||
"tyIMxz73w1vx/NUQ4TIRWloZcT+ipQKvNuv5SyTVUPc7XOCb41Hg9horQq0HRtepMFkrTXIWlcWU\n",
|
||||
"NmBivQ6rBTM/9dgoEoHfTiLwi7Bocxt0LI3YlE0jgoSJWa0i1qUZnto2Brvmg0kEvjsj+Gj1NCLw\n",
|
||||
"d9H9EfzbyB9If84TLJx/pvhCP0XO+ckWTtrqdbjAN8cjWBGhXFTZhRUnqiVGI6gU+HSjMJNkIk58\n",
|
||||
"7JiwbRE2oSrWEclG8EMwgV+IRajpRqgpgcemqf+47l7FacOsrrTAxwheSKyp7hT4IZn/9RiOfe4H\n",
|
||||
"ijS8ZGAhQq9iAHCGSOFsrOcUWtLPF00VdQoIvIj8QERWi0juEnoiMl9ENonIwvBXZMCqr/Nx8lcO\n",
|
||||
"SrMeas5qrYjgQ6OwAxPnvAj+WCyr5llsZuUR4bE2kog3G8FDawT+NJpYMq4GMa1zOHTU9IlTtDeR\n",
|
||||
"vJ703IDcQdYu9CyGZv7XI35eD9J9Ufwg7DvwaZJVkRynaYpE8D/EUpNqcYOqzgt/n23BdfVqVFmt\n",
|
||||
"WndN1mzaIwAinCbCyXSe6AQWsU4gP4I/nET0nyZZuGIMiSDuwCLcYVg0DJX53c0K/Imp52sFbeFa\n",
|
||||
"YgQ/ClifspHi69mS2mcwVoM8G8HfK1KxFF9RmrFoWiLwInxAJHfG8xDs83qQrpVkcByggMCr6l+h\n",
|
||||
"Itc7j2aWaNvfiamNWd6EpWVlPXiw9Lu3kR/B9ycR/dXQMRA3iWRB8e2YMGwPx++lMoJfSYMCL8JQ\n",
|
||||
"TNBaMvAXLKiRVAr8aJKUMVvtykgL/BBs0DVZ1s0Kjx3S5LU1KvDDSAS+qwOtp2NjKlmiwG+htdlC\n",
|
||||
"znOUVnjwCpwsIveIyNUi0q1ZBn2I3AgeE9iTsYlSWYH/HLYk2GwqBT4KXty2ikqBj5koO8L2Tars\n",
|
||||
"xrINumrRHIfZPSLSEtEZio1PbCAR7zEkr/EJCgo8SeXAWouJ1LqOeN4ixAb5UZLl85plRJXnjQK/\n",
|
||||
"lcrxCcdpilbkwd8NTFPV7SJyDlZNLXdNThG5OHV3gaouaMHz91aqpUpOw1bBuZCMwKuyQoSfAW9S\n",
|
||||
"JT2yH3tQMYJfRbIy0WQqI/hJJCmR11OZUtaMwJ+IlSiegDUeS2vvniDCcaqdClPFQeG0eI8mEfVL\n",
|
||||
"sAYA7PUMFOEAzKJZg5VYjZyO9VKaFfjtNC7wT2auoRmGV3lej+AdRGQ+lZlyTdNlgVfV1EChXiMi\n",
|
||||
"l4rIaNXOlfRU9eKuPl8fomOQVYS3Ar9Q5RlMYF+DVTXMevBgEy+yNSzWY/5zTA+rG8EDqPL2zHlW\n",
|
||||
"YBk4Eh4vMuHpJKww10mYyBcS+CDKt4gwVZX0qko1BV41mYauioqwFRO7GMGnF7uej+UwNyPw8Xzd\n",
|
||||
"KvBh0trWMDgeKRTBNzkpzenjhMB3QbwvIqWqO9ehyxaNiEwQsUL5InICIHni/hykHRgTxPTbwFwR\n",
|
||||
"RmBe+iNYUbLO06ltAPermc1PAQ+mRGI1MDGcOx3BR4HPazgIA8PPYEWrltTJ049++YlYGeR0o1KE\n",
|
||||
"GVgAkU0prBfBZ4n7VVg0IkwJ57qZggIfqnHGBZKH0pzArwLGhgasCD+gcj3ReK5qAr8j2GvPQkUt\n",
|
||||
"fMdpmCJpkr/ASmEeIiIrROTtInKRiFwUdnktcJ+ILMJWur+w+y63TxEHWcdhP9xDseh9RZgZe4fa\n",
|
||||
"Itt1UbU1H1ObotgOB/aG2bXQ2aLJYwW2fuUU6Milr8Y8LJpcgTUqjWTSxJLK2XGIRgU+G8EPCw3P\n",
|
||||
"6dhM4vXUKYOcYiYmuGACny7QVo/hJJH4Ooo3dvOw3lqaahH8YJKsJ/fhnS5TNwpR1ZolSlX121iE\n",
|
||||
"6lQSB1lnhfuHYiK5suoRNch01ddgkfE0EnsGkgg+d85C4Nck67AeFK6pGhcAvwlWySoaE/i4uHVR\n",
|
||||
"ga82OzHuNzjssxebDDQXKwu7kcrFmmsxHBgVSg/HCH5eA8fGhjTaNDU/y7B4yQRsAttoVdaHxim+\n",
|
||||
"nizRooGkYVubs5/jFMJnsnYf0YM/CBPeQwgRfFdPHLrwG7EVbp5OPbSdlAdf5djPqvJ7zOc/qNp+\n",
|
||||
"QYguICmalk7NLEIjAp/OosmStmi2Y1lBQ8O1PIW91qIe/HDoqLY5FIvEG53oBMV9+COxErTXkyzI\n",
|
||||
"PjRcQy0PHirfH8dpChf47iNm0czCfuAdFk2Lzr8aiz6zEfwgals0kWUkvYs8jsa+H7Fee0cEL9Kx\n",
|
||||
"+HEHIpyQ8fTnYAOyRT34aqVT0wK/gySynYg1bhspLvAxM2U0zXvwUFzgj8J6U78DXhW2xWJv9QQ+\n",
|
||||
"vk7HaRoX+O4jWjQHYROYDsIEtVUCvwqrT5MW+GzlyFrUjOBJ2TPhfhzYnQjcIZIsHiLSsYZmevxl\n",
|
||||
"dtjWVQ8+bdHEDJNh2FjDKhoT+PTzDQnHUmUZxA6CpTOEpPTyUzQm8FcBZ4WB2XRefxaP4J2W4gLf\n",
|
||||
"fWzEBv9mY930J7GBwVYK/DwqLZq4TFwrIvgzgaszzzcBW7oQbKJWTAP8DlY75SsitAVBPAhb2abV\n",
|
||||
"Fk06go8CPzJcy7w6q2nFiHgUyXKGRXLhhwLbU4PiDUXwqqzDspdG4RG804O4wHcTIdtiCzYYuAxb\n",
|
||||
"ULiVFs0qTBhbFsGLWI55KOB1FJWLbEcP/kxM4E4M28vAlap8EVvY+HOY+G3AJlnFuQBfEOF9ZAQ+\n",
|
||||
"FBobT/XlALMWzbZwjjbMQ09H8G+kdpGudATfiMBny0rUFfhgV8UIHpLCacPD7appkuF2RwMo0r01\n",
|
||||
"6J3qiPAGEd64r6+jWVzgu5eYwvcEJvDQWg8emo/gV2CWywAAEY4EHgoWzFHAY+mCaqH2/bPAy7Fa\n",
|
||||
"Oi8IlsMbSOqX/zNm05yN+e/papBHAx/FBD8dwc/BFjepWGQ5RV4EPxtYq8qe8FrbgqBOwUrtVssO\n",
|
||||
"6zGBxywkJfmcYunjEVjjXCRNMkbwN4e6O07Pczz23e2TuMB3L+1Y3vsuTOA3pXLWu0pc6i4vgs+d\n",
|
||||
"6JQmCOpT0CEccWHts7Ev9R05h63CMkB+hPn/Z2Cvb1k4ZzvwLUzwo8BHi2YGJmCnYgK/E0vTPZKk\n",
|
||||
"8csjz4OfE64lllnejQnmVEwUj61yrmHYjOBGLZpcga8zUSzaM3EMIwr8cKoLfCcPXoSBWA9lUp1r\n",
|
||||
"3KeIMFIk+RxFGN/AZLDeTByv6ZO4wHcv60nqxzxAZQngrhIFvtkIHoIPH4TqQsxLfylwAuafZ1kN\n",
|
||||
"XK/KJmxl+zLw35l9voo1AksJq1SF888ALsZEfWMQvi1YMbNOM3pT5EXwc6h83dGmmYKNG7y4yrmG\n",
|
||||
"Y+KcjuC30bjAxwa01vKHp2AzgCPpCH51fE4RThThv8I+eR587AHtE4EXYYgIRUqAzwQOTjV6P6L6\n",
|
||||
"59CXcIF3qtJOUlfmFiw6bhWrsJmVaeFpVOCjD38cNoHoEuwaX0B+BP8E8Odw+1asPs1v0zuEImlv\n",
|
||||
"CdvXY8I7FtiFzSK9mcS22IL1FopE8GkPviOCD2zEovLJ2MpTtQT+cZIf7fbwVy8XPrs4i5Jj04gk\n",
|
||||
"lhfWu7k+9XBckrEjgk+t3hUnauVl0cS68Psqgp8DfCTbWxHpVEhvKqYncQLXqPDX13GBd6qyBisv\n",
|
||||
"GxfubuWsxEeA7OpZjQyygkXwLw3n+aUqK7DIeA75s2HfiUVmYAJ/vyoPZ3dS5SpVHg4DzdswD/Nx\n",
|
||||
"VXapcmqqUdqCWT1FIvi0RTObzgI/OzzXH4HjRDoi5GkiHWUThmGNVFWLJmTiZAU/r3b/Sjqnmf4Y\n",
|
||||
"+GB47nlYYxaJ4xEjsIZPsVWs2kiEsFYEv68W4p6MvfcdvZUwHrA0I/oxbXZ46n9FmmemAewrjCJ/\n",
|
||||
"1nGfwAW+e/ks8M3uOLEqO1X5embzDpJiVUX4C8lCIrHcxB+AhcHbzj7ntjCwCfAz4PwCz9GOifjj\n",
|
||||
"OY/Fsri1BD7WZElPdBpIZ4vmSGBlGBi+FTgvPPYV4MPh9nBM4HMHWUM+/B/pnIkzHDqt4PVj4Asi\n",
|
||||
"9uMPYncsVs//VGBRZlH2tAe/OfW8I8kX+KYieBEmiXBJkX0LMjnn+edi15we+I1lqOPAcCeBx4KD\n",
|
||||
"l7fw2lqKCB8V4SOZzR7BO/mo0q5af8CzhawjVWa0Hqrcqsp5qnwoVdL3P4EvFDh2e170nkM7Fs1W\n",
|
||||
"E/h1YXC2GlswkdsdGpcotNkI/kiSVMvLgPeLMAkr9BUHetMWTV4E/zLsPXxPxoLIi+B/go2rxAyi\n",
|
||||
"KdgA7k6szs/1mf3THvwWMgKfmkwVbbYYwY/DMp6KWjRHYuWoW0UU+HQPIha+OyK1rW4Ej/WyevNg\n",
|
||||
"8fNI2Xuh0W5I4EUYWq9Ka0/iAr8focoWVc7t4jmWqPK7Vl0TJmzHULnwSGQLtf33uM94ksg2RsVp\n",
|
||||
"gd+EiU0s/nUFJjjfDM8bbY5o0UzEqnDuplLg34algP4v8KHU+eNyfR0EH/4i4M0ijMcasYXAd7EV\n",
|
||||
"u/IEPnrw2Qi+H5U2VHzdMYK/j+IWzURqD/42Sl4E/3zs/U/n53cIfKqgWofAhwZsBq1d27fVjMHS\n",
|
||||
"f6NAD8YK2zUSwV+Nfd97BS7wTnfTjkVG1SL4WvZM3Gckld40dLZoDiUIfPD+L8MspEuojOCj4MeG\n",
|
||||
"Yjs24DkROA0rrnYJ8N7UDz0vgo8DytcBL8Fsi4XAT7ESDbdkdq8VwYNZHtU8+PsoHvl2h8BnexBH\n",
|
||||
"YVVJ0xH8NOy9HYZZaOmyDITjD6T3C/xokkJ5sRfXiAc/kfylOvcJLvBOdxPtlzyBXwPcW+f4dK17\n",
|
||||
"SAQ+XeZ4IyYe6dmw3wU+jWX9xAg+ZrDsIiPwmJVzhSpbVXkUs1smpo6rNn/hGuAcLIJfpMoGVU5U\n",
|
||||
"7bBaImmBz0bwUJnZE193jODvB8aHKLgeE7EGq6Ec9ODd51kLk7AZzRPDfoOxSPx/CQIfjpuKLUie\n",
|
||||
"jtzTAh/LYvSYwIswQoRzGjhkDPAwySzt0dhn1UgEP5JeVGLCBd7pbmoJ/D8Dl9Y5ficmtmlvemt6\n",
|
||||
"li10rF/bUZ9dlfWqfJbK2bTRatlAZ4E/ksrSDA+RpC/WE/izsQHWhVX2AbORhmGikY7g27CJWtUi\n",
|
||||
"+HFYw7WZYpFhFNBGC5XdTP4EscnYussxgj8cE8FFwGGh0WnDZjk/Fa45Pnda6GZiEX5PRvDnAp9p\n",
|
||||
"YP+xWGG4WM9oFPbeu8A7ThVioa1OKaIhbfLZzodU7BMnRKVn6T6V2S0KfF49m21Av1AULc58XU+l\n",
|
||||
"pz8EOAyLQCNLSAQ+2ip517cyXM8YaqxXGwqVbcAyT2IEP5iklMVYzO+NC46nI/i1mCVVxKaJvY7C\n",
|
||||
"No0Is7AIe1Jmez9MkBemzvt8bIbuxvA6poW/lalrzovgZ2LWVU8K/HEUbOhCL2QM5qGnI/jCAh9S\n",
|
||||
"QAdRfI2BbscF3ulu2oEnii5PWIW0wN9HkgIZiXn/nVZYCg3EOkxgokivpzKCH4qJeXrA9yHg0PDD\n",
|
||||
"PxqrCFqNa4B7CrzGdZiVlPXgl2NZONtTpQ22Y172REzgV1Fc4PfQmA8fK4SOy2wfhzWeT6Se+/kk\n",
|
||||
"tjqGaqcAABLySURBVNoDmE0zFXvvY0rrcKyhyhP4estEtpJjybwPYd2CdhE+GVNcA0Ox9+1vWM9k\n",
|
||||
"MInAD8qzr3Iss2i3eQTvPGd4CMu37wpbCBaNKntVOw3MbsQEu9oEr3YqBT5r0UzCxCjdQCwhWYXr\n",
|
||||
"AGqXmfgeFMo9Xxf+b6VS4JdhVkiHbx+EfhtmE6zHIvgimTQTw/kaFfh2OotvXNA93XuYS77AryCZ\n",
|
||||
"1zAc69VkPfj7gf45E8laThDfY+gcwZ+Drc9wHvC+1PYxQHsYO1mCWXajSHqgg3Ke5gYRHhLhg+G+\n",
|
||||
"C7zz3EKVe1V5dxdPk47g81gJ3JZZtzbNOkxgom+fjeCPBZZkjo8e/EnALTXOjSpLVbmq7quw64iT\n",
|
||||
"xbaTiOHjhAg+s/9WYENI56xq0YgwV4SDw0St2GAUEvgQmZ6B1RTKRvCTMaFux9IfR2C2R6yxczNW\n",
|
||||
"onkOnS2arMDPDNe1GpggwgHhfN3FbMIAaSbSPgObw/DLcE2RsSQN8IOYZTcaCwaqFaSbAXyeZEZ5\n",
|
||||
"FHi3aBynAWoKvCorVTmzxvH1LJrxVPrvYLbJBOBFdE55bJZ2kkJlO0jy+9diUXD2NW4hGbuo5cH/\n",
|
||||
"G/CecL61mCgVFc/ZWHG4v4XjSdkRk4GngvW0Bot6HwzF5sAahaXAB7AIPm3RdAi8CP3D63ucIPDA\n",
|
||||
"m7BJdZ0Q4VSRQguq1OJYrJ5SbEgJaw8cB/w1XF/6/RxDkhAQG/e40tgO8gV+DHAlNlGtP8m6BB7B\n",
|
||||
"O04DREujWeoJPGQmXIXB38eA19M6gV9HZdrnJMxW2kD1CD4K/CpyLJpQTvgMLE0zrnK1meICfyY2\n",
|
||||
"KWsNSQT/DhG+E64vDmg/jUXrHRO4Qq/m77HofQmVFs0qYHCInidjM5afIRH440nyzbN8HGsAusJx\n",
|
||||
"WFZUeuWwk7D6SVvC65qc2j9P4KM91imCDx79gSRrG7ThFo3jNEWHB98k7VRaNDeSjAvkCnxq21Dg\n",
|
||||
"zi48d5p1JBH8dkxgNmEikifw6Qj+CeDwnMG+UzCxmhfOV1jgg/i+H6srtJZE4Odhs3RfSSLwq4Cz\n",
|
||||
"sIldHYRsmkNVuYlKiyZmCg3D3vvl4ZAo8PMwiyOPcVjJ6q5wLPa5bSYR+HSFzzyBjxbNQyQWTRT4\n",
|
||||
"7GSnMVijFQfxx2ACv5m+JPAi8gMRWS0iedUF4z7fEJFHROQeEZnX2kt0nLoefD0qInhVblDlN+Gx\n",
|
||||
"eN6sRQP2Q1+k2qXnzl5HtQh+APkRfBSdW7Hfa7YU8kuAX2DCciImoEUj+FdjDecfMIGPg6wzsHLP\n",
|
||||
"6UXdn8YqYN6cOUfs7cTrjQK/hUTwZ5DMg1gTXvfzgWEiuWmM4+m6wB+OZVxtIXkv0gK/ClvRLGpg\n",
|
||||
"OoJ/BGuUJlDdg0/vHyexjcSybvqUB/9DrKRsLiJyLjBHVQ/GKuld1qJrc5zICiprzzTKOqpPVtqG\n",
|
||||
"pcfl5bD/gSo+cZPch/m/0Fng47Y0HRF88MEvAT6V2eclWAXMhViGSKEIPvQEPg18JkSha4FxqcVZ\n",
|
||||
"PodF9jFjZhVwe2aCWZa0RZMW+MkkcxRWY72O1ZgFlhfFjwNGi1RE2IVJFQlrp9KiOQyboEWwizaT\n",
|
||||
"TIIbG/ZHlZ2Y7XQ4VSwaKgU+TqaLAt93InhV/SvJFzCP87DSqajqbUCbiPTmehNOH0OVL6jyrS6c\n",
|
||||
"Iv4Q88RpFXBRlfLIf1Xl8i48b/Z8d6tSCne3YxHiRkxEoLMNtRmLeCO/BGaIcDJYeQFs4tTtmMDP\n",
|
||||
"o7hF8wIsz/6qcG3bsdmoMeJersqb4nKMwLXUL32dtmi2kMzGjemWYMJ+Wrje5WQEPowpDMFstGaj\n",
|
||||
"+CFY9dEo4iNSM27Xp/ZLD7SmBRus99af6oOs6aybtEXTtwS+AFOoXEh6JUllOcfpDcQfYl7BsGdV\n",
|
||||
"+X4PXw+YwPfDIvjNmP2RjeAvxnrQQIcV8j1seUUIA6Rh+91hW1GBfzFwdSb9cw1WGE6Dt96BKjeq\n",
|
||||
"8ss656xm0aQHa1djdtRCzLaZmTlHFM5baV7g20hmN8drGImVuEjPnE778GkPHsyyUyrrBqWpZtE8\n",
|
||||
"RS+yaFq1KG524Cc3Z1hELk7dXaCqC1r0/I5Ti6oCvw/pWH1Llb0ibCQj8KoVFTMj15KI/otIBotj\n",
|
||||
"HZxV2KScegJ/JlYaOc1aLPskr25QEaJFE0s7pAU+HcHH691DiOBFGKXKBsyeWYP1Sj7a5HWkBT4O\n",
|
||||
"smYjdOgs8NkIfkP4bKoNsuZZNHfTxQheROYD87tyjkgrBP5JktVcwKL3vJogqOrFLXg+x2mUWhbN\n",
|
||||
"viK7vGK6Pk4t7gamiDABE+m44MgKTKCfxESyqsCHJQWPx2yQNF0SeFV2iaCY+GU9+Cjw0XJaiAni\n",
|
||||
"MSKMA5aLMJak9s4dwPEiDMizz+qQjeBHkGTEpKkl8A+m9o8lpQWYptpROyg6F+uw+QRttMCiCYHv\n",
|
||||
"gnhfREpVd65DKyyaK7BFlhGRE4GNqrq69iGO06Nsw6ab98oIPvzfQAGBD7Ngb8Dyzw8k1NMPVstc\n",
|
||||
"VZaSsmhE+IecGaNxScFsg7cGE/5mI3iwRnQy1SP49diYx9MkHvw5mAUykyDwqqzDBqSbieLzLJp6\n",
|
||||
"EfzYzOO3Aq8Nt6NFcyhJFlHa0klbNKuw2jX9m7jullMkTfIX2Cy3Q0RkhYi8XUQuEpGLAFT1auAx\n",
|
||||
"EVkKXI7NqHOcXkMqV7k3CnwUokICH7gO+BhwXdpDV+3wuePAogBfxPLZ05yJWT1Z1mK1Zboi8Fuo\n",
|
||||
"jOCnAnviQuth8fnvhn2jB/9yYC+WmhgjeLAc/Y+KdFrcHAAR/l2EN+Y8lLVoRpAv8E8Dk0MVyIEk\n",
|
||||
"cxRizaN7wt04yDoZmCrCMKpbNLEuUq9Yx7WuRaOqbyiwz/vq7eM4+5h17B8WDZjAf53MpKMUUdTG\n",
|
||||
"h/+vxOqvEFauuoDQ686wBssc6arAx/9bsAYjbywBLNodiaV6XoEJfCy3gCrLRPh3bKzg/Jzjz8WE\n",
|
||||
"9WeZ7XkRfDWLZhIm1utr1Bvajs1qjfME5tB5kDVm0WwiyR6qGVCEBni+aqflHVtGqwZZHae383Ys\n",
|
||||
"D723kBX432B54UV4AKvMmBeFQyIwh2CTds4KdVhmYLn9PyBnwhJJ5NxViyb+34Jl5WTr9wMWJYuw\n",
|
||||
"AhPeWzGBH0GSEQRhXV0RpgfvG7BBWez15bkQjVo0B1GZQZMlzjqO6d8H0zlNcmx4nhjBF/HhZwN/\n",
|
||||
"FGFgrWJ2XcFLFTjPCUIO+u59fR0pKgRelf9WrbkiVAfB5jgqLXiZx2O1ynlYHZ17sFmrVwGfV+Xf\n",
|
||||
"qghKKwR+C1bXfg+JwFeL4MF8+Kuwxi1aNB25/6psw3of/5A57kSskZos0rF2aqSoRbMKi8r/E/hK\n",
|
||||
"jWuMHvx4bK7A8zLnW4/1EPphK5BtpUqqpAjDwqAy2Nq2B9LaNXQrcIF3nH1DnNRUrYZ9V9mMZcQ8\n",
|
||||
"DPwfllr5J1W+V+OYNdhg9Joa+9RjK0kUvwVLL8yN4AP/io3dLcMi6bQHH7kMK4A2MLXtJOAmrN7M\n",
|
||||
"C6BjkhQUtGhCg78BuEk1mW+QQ/Tgx2MFzA7HIvSN4TzPYu/3ptBwxh5UBeH6rgG+HTYdGf5nyzS3\n",
|
||||
"DLdoHGcfoMoeEbZRe5Z4V4gCfyUWxc8FPlLnmEeB/2rB6ltpHx5qRPCqVqlThD1YBL+ajMCrskSE\n",
|
||||
"e4HXEcYSMIH/Ojbn5sQwEPtKbH3cohYN2HjE7XVeU4zgh2ONyqsIOfKpfdaRzAfqZNEEv/3SsM8p\n",
|
||||
"4f5R4eFx1FjusSt4BO84+46jVJPMjRazGfOoH1ZlhSpvDlP3q6LKRlXe1cXnTQt8jORrWTSR9Zge\n",
|
||||
"zSRn/V7gq8BHRJCQgngC5tvfimXhfA6zTiDfoom1aSoIM3R31rm2ONFpPCbws3PO1U7SG8uzaE7A\n",
|
||||
"Jqa9FAusp2ER/EqSejgtxwXecfYRqTov3cFmLFrslsiwBnFwldT/WhYN0JHKugzL4snr1fwBS2V8\n",
|
||||
"EVblcnXIlb8t3P8B5sfHhTfyIvhsFk1RYgQ/gaRCZVbg11Ep8FmL5u3Ad8Pcg1uwmaqzsFx/t2gc\n",
|
||||
"x2mIzdhqTD2dGtqQRZNhGTAxzyIKGTdfwQZDJxKWyVNljQgfwgZK34iJcFbgh2GNXZ5FU4S4MPt4\n",
|
||||
"zEJ6OOdc60gW/KiwaMLM4QtILJm/YYL/GMnM4wrC5LTRqh119JvCI3jH2T/ZjAlRT9NVga81wPsz\n",
|
||||
"LPPlDapJGWdVvh6ybVZg1keHwIdsnp1YBN7sgPYOTIQ1NJiP0DmtslYEfz62rm8s4XILcDrWG+hY\n",
|
||||
"aEWE14rwcxHuxno972zyejvwCN5x9k/2lcAvhI7aMRtJVlUqwjLy/Xego0571bUpqBT4tJjH1M1m\n",
|
||||
"c823Y1VzY/roYqygW5rVJIOsW4GhIhwJfAZ4IVZaInInlm55P/Z6Dw/bP4FlPH0TuLvemEkRXOAd\n",
|
||||
"Z/9kAfR83r8qfyUsahIE+fgGDr+RrrkKK7CB5d0ZcdwMXcoM2o6Jd+xdfJnOFXQvJdHTrViJhndi\n",
|
||||
"kf0LVZMlIVXZIcKd2OIjQmLRHAR8X7VLi9tU4ALvOPshqvzvvr6GRlFlEWHFpSZZga3FujGzfQvU\n",
|
||||
"zZSpRZyUtho6Gq4KgkUUiR78kcCn0uKe4mXhOk/AVtIagdlILS3U6B684zj7Cyuwgcw8gW82gwaS\n",
|
||||
"SWlFJ4BtxWyio7GJUZ1QZX0YTI4e/CxgWatLFrjAO46zvxAtmqzAb6b5DJo4ULuL4tH1VqyUwuOx\n",
|
||||
"imYN1mJ58LMoXouoMC7wjuPsL6zE8ujzIvimBT6wneIR/DZssPeOAvtuwZYw/P/t3VuIVVUcx/Hv\n",
|
||||
"r9IHMwgJxi4D+uDD+OQQDJFI8yT60oWiFAIfeoju0EMiSPrQgwVBD0EEGViEJUViEGRBRRAkkrdS\n",
|
||||
"KcEBLS8DRSQSKP17WOvk8Xgue2b2OXtm+/vAxj1775mz/LP8u2fv9V9rBMqvi3CCN7O6+J00dUHZ\n",
|
||||
"j2hgagm+UXvQM8E3rVUwhu/gzczay5OHneHaBP8ezPil80Wm9ogGes9x0zBJmjCt9ATvUTRmVien\n",
|
||||
"aEnwEXxfws/9hrw8YgEXSENUD/W6MJskvZAt/RGNE7yZ1ck1Cb4MEVOqKj0FPDyFQqVGVawTvJlZ\n",
|
||||
"Fx+RXrZWJo+6+WwK3zJJWmi89HmDnODNrDYi+LjqNkzDJH14/g5+yWpmVrWz9GlaZ0X0Za3Xaz9I\n",
|
||||
"iohonb/BzOy6lhdEXxDRfijnTHJnoTt4SWskHZf0q6SNbc6PS/pL0oG8bZ5OY8zMrjcR/NMpuc9U\n",
|
||||
"zwQv6UbgTdI0ncuB9ZJG2lz6bUSM5u2VkttpLSSNV92GunAsy+V4zh5F7uDHgBMRMRERl4APSYvb\n",
|
||||
"tvLjl8Ear7oBNTJedQNqZrzqBlhSJMHfSRrX2XA6H2sWwL2SDkn6XNJyzMysUkWGSRZ5C/sjMBwR\n",
|
||||
"FyWtBXZzZYVzMzOrQM9RNJLuAbZGxJr89Sbg34h4tcv3nATujog/mo4NZriOmVnNTHcUTZE7+P3A\n",
|
||||
"MklLSLO1PQasb75A0hBwPiJC0hjpP46r3gp7iKSZ2WD1TPARcVnSs8AXpLmWt0fEMUlP5vNvA48A\n",
|
||||
"T0m6TJp1bV0f22xmZgUMrNDJzMwGayBTFfQqlLLuJE1IOpyLyPblY4skfSnpF0l7Jd1adTtnK0nv\n",
|
||||
"Sjon6UjTsY7xk7Qp99XjklZX0+rZqUMst0o63VTouLbpnGPZhaRhSV9L+lnST5Kez8fL6Z8R0deN\n",
|
||||
"9FjnBLAEmEdaNX2k359bp400jeiilmOvAS/l/Y3AtqrbOVs3YBUwChzpFT9SMd/B3FeX5L57Q9V/\n",
|
||||
"h9mydYjlFuDFNtc6lr3juRhYkfcXkuacHymrfw7iDr5ooZR11/qS+n5gR97fATw42ObMHRHxHfBn\n",
|
||||
"y+FO8XsA2BkRlyJigvQPaGwQ7ZwLOsQS2hc6OpY9RMTZiDiY9y8Ax0h1RqX0z0Ek+CKFUtZdAF9J\n",
|
||||
"2i+psfDAUEQ0lhA7BwxV07Q5q1P87uDq+cTdX4t5Lhc6bm96nOBYTkEeqTgK/EBJ/XMQCd5vcWdu\n",
|
||||
"ZUSMAmuBZyStaj4Z6Xc3x3maCsTPse3uLWApsIK0JurrXa51LNuQtBD4BHghIv5uPjeT/jmIBP8b\n",
|
||||
"MNz09TAVr7gy10TEmfznJGnx4DHgnKTFAJJup/iK75Z0il9rf70rH7MOIuJ8ZMA7XHlk4FgWIGke\n",
|
||||
"Kbm/HxG78+FS+ucgEvz/hVKS5pMKpfYM4HNrQdICSbfk/ZuB1cARUgw35Ms2kKaHsOI6xW8PsE7S\n",
|
||||
"fElLgWXAvgraN2fkBNTwEKl/gmPZkyQB24GjEfFG06lS+mffl+yLDoVS/f7cGhkCPk39gJuADyJi\n",
|
||||
"r6T9wC5JTwATwKPVNXF2k7QTuA+4TdIp4GVgG23iFxFHJe0CjgKXgafznanRNpZbgHFJK0iPCk4C\n",
|
||||
"jSJIx7K3lcDjwGFJB/KxTZTUP13oZGZWU16T1cysppzgzcxqygnezKymnODNzGrKCd7MrKac4M3M\n",
|
||||
"asoJ3sysppzgzcxq6j+vUsbacqJa4gAAAABJRU5ErkJggg==\n"
|
||||
],
|
||||
"text/plain": [
|
||||
"<matplotlib.figure.Figure at 0x7fbb37f207d0>"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"plot(np.vstack([train_loss, scratch_train_loss]).clip(0, 4).T)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's take a look at the testing accuracy after running 200 iterations. Note that we are running a classification task of 5 classes, thus a chance accuracy is 20%. As we will reasonably expect, the finetuning result will be much better than the one from training from scratch. Let's see."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Accuracy for fine-tuning: 0.570000001788\n",
|
||||
"Accuracy for training from scratch: 0.224000000954\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"test_iters = 10\n",
|
||||
"accuracy = 0\n",
|
||||
"scratch_accuracy = 0\n",
|
||||
"for it in arange(test_iters):\n",
|
||||
" solver.test_nets[0].forward()\n",
|
||||
" accuracy += solver.test_nets[0].blobs['accuracy'].data\n",
|
||||
" scratch_solver.test_nets[0].forward()\n",
|
||||
" scratch_accuracy += scratch_solver.test_nets[0].blobs['accuracy'].data\n",
|
||||
"accuracy /= test_iters\n",
|
||||
"scratch_accuracy /= test_iters\n",
|
||||
"print 'Accuracy for fine-tuning:', accuracy\n",
|
||||
"print 'Accuracy for training from scratch:', scratch_accuracy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Huzzah! So we did finetuning and it is awesome. Let's take a look at what kind of results we are able to get with a longer, more complete run of the style recognition dataset. Note: the below URL might be occassionally down because it is run on a research machine.\n",
|
||||
"\n",
|
||||
"http://demo.vislab.berkeleyvision.org/"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"description": "Fine-tune the ImageNet-trained CaffeNet on new data.",
|
||||
"example_name": "Fine-tuning for Style Recognition",
|
||||
"include_in_docs": true,
|
||||
"kernelspec": {
|
||||
"display_name": "Python 2",
|
||||
"language": "python",
|
||||
"name": "python2"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.9"
|
||||
},
|
||||
"priority": 4
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
31
Deep_Art/deepdream/examples/CMakeLists.txt
Normal file
31
Deep_Art/deepdream/examples/CMakeLists.txt
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
file(GLOB_RECURSE examples_srcs "${PROJECT_SOURCE_DIR}/examples/*.cpp")
|
||||
|
||||
foreach(source_file ${examples_srcs})
|
||||
# get file name
|
||||
get_filename_component(name ${source_file} NAME_WE)
|
||||
|
||||
# get folder name
|
||||
get_filename_component(path ${source_file} PATH)
|
||||
get_filename_component(folder ${path} NAME_WE)
|
||||
|
||||
add_executable(${name} ${source_file})
|
||||
target_link_libraries(${name} ${Caffe_LINK})
|
||||
caffe_default_properties(${name})
|
||||
|
||||
# set back RUNTIME_OUTPUT_DIRECTORY
|
||||
set_target_properties(${name} PROPERTIES
|
||||
RUNTIME_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}/examples/${folder}")
|
||||
|
||||
caffe_set_solution_folder(${name} examples)
|
||||
|
||||
# install
|
||||
install(TARGETS ${name} DESTINATION bin)
|
||||
|
||||
if(UNIX OR APPLE)
|
||||
# Funny command to make tutorials work
|
||||
# TODO: remove in future as soon as naming is standartaized everywhere
|
||||
set(__outname ${PROJECT_BINARY_DIR}/examples/${folder}/${name}${Caffe_POSTFIX})
|
||||
add_custom_command(TARGET ${name} POST_BUILD
|
||||
COMMAND ln -sf "${__outname}" "${__outname}.bin")
|
||||
endif()
|
||||
endforeach()
|
||||
BIN
Deep_Art/deepdream/examples/bgk1.jpg
Normal file
BIN
Deep_Art/deepdream/examples/bgk1.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 102 KiB |
BIN
Deep_Art/deepdream/examples/bgk2.jpg
Normal file
BIN
Deep_Art/deepdream/examples/bgk2.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 4 MiB |
BIN
Deep_Art/deepdream/examples/bgk3.jpg
Normal file
BIN
Deep_Art/deepdream/examples/bgk3.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 538 KiB |
BIN
Deep_Art/deepdream/examples/ctl1.jpg
Normal file
BIN
Deep_Art/deepdream/examples/ctl1.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
BIN
Deep_Art/deepdream/examples/d1.jpg
Normal file
BIN
Deep_Art/deepdream/examples/d1.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 517 KiB |
BIN
Deep_Art/deepdream/examples/d2.jpg
Normal file
BIN
Deep_Art/deepdream/examples/d2.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 138 KiB |
BIN
Deep_Art/deepdream/examples/d3.jpg
Normal file
BIN
Deep_Art/deepdream/examples/d3.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 56 KiB |
BIN
Deep_Art/deepdream/examples/d4.jpeg
Normal file
BIN
Deep_Art/deepdream/examples/d4.jpeg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 238 KiB |
BIN
Deep_Art/deepdream/examples/d5.jpg
Normal file
BIN
Deep_Art/deepdream/examples/d5.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 196 KiB |
BIN
Deep_Art/deepdream/examples/d6.jpg
Normal file
BIN
Deep_Art/deepdream/examples/d6.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 939 KiB |
BIN
Deep_Art/deepdream/examples/d7.jpeg
Normal file
BIN
Deep_Art/deepdream/examples/d7.jpeg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 531 KiB |
8392
Deep_Art/deepdream/examples/detection.ipynb
Normal file
8392
Deep_Art/deepdream/examples/detection.ipynb
Normal file
File diff suppressed because it is too large
Load diff
537
Deep_Art/deepdream/examples/dream.ipynb
Normal file
537
Deep_Art/deepdream/examples/dream.ipynb
Normal file
|
|
@ -0,0 +1,537 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "AeF9mG-COZNE"
|
||||
},
|
||||
"source": [
|
||||
"## Loading DNN model\n",
|
||||
"In this notebook we are going to use a [GoogLeNet](https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet) model trained on [ImageNet](http://www.image-net.org/) dataset, which is set in the ```model_path``` variable:\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "RMhGdYHuOZM8"
|
||||
},
|
||||
"source": [
|
||||
"# Deep Dreams (with Caffe) adapted by marina von steinkirch\n",
|
||||
"\n",
|
||||
"This notebook demonstrates how to use the [Caffe](http://caffe.berkeleyvision.org/) neural network framework to \n",
|
||||
"produce \"dream\" visuals shown in the \n",
|
||||
"[Google Research blog post](http://googleresearch.blogspot.ch/2015/06/inceptionism-going-deeper-into-neural.html). **#deepdream**\n",
|
||||
"\n",
|
||||
"## Dependencies\n",
|
||||
"\n",
|
||||
"* Standard Python scientific stack: [NumPy](http://www.numpy.org/), [SciPy](http://www.scipy.org/), [PIL](http://www.pythonware.com/products/pil/), [IPython](http://ipython.org/). \n",
|
||||
"* [Caffe](http://caffe.berkeleyvision.org/) deep learning framework ([installation instructions](http://caffe.berkeleyvision.org/installation.html)).\n",
|
||||
"* Google [protobuf](https://developers.google.com/protocol-buffers/) library that is used for Caffe model manipulation.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"id": "Pqz5k4syOZNA"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"import numpy as np\n",
|
||||
"import scipy.ndimage as nd\n",
|
||||
"import PIL.Image\n",
|
||||
"from cStringIO import StringIO\n",
|
||||
"from IPython.display import clear_output, Image, display\n",
|
||||
"from google.protobuf import text_format\n",
|
||||
"\n",
|
||||
"import caffe\n",
|
||||
"\n",
|
||||
"# GPU support for CUDA and Caffe.\n",
|
||||
"caffe.set_mode_gpu()\n",
|
||||
"# Select GPU device if multiple devices exist.\n",
|
||||
"caffe.set_device(0)\n",
|
||||
"\n",
|
||||
"# Set the background image to build the dream on top of it.\n",
|
||||
"BACKGROUND_IMG = 'd2.jpg'\n",
|
||||
"\n",
|
||||
"# Set the image to control the dream on.\n",
|
||||
"CONTROL_IMAGE = 'd3.jpg'\n",
|
||||
"\n",
|
||||
"def showarray(a, fmt='jpeg'):\n",
|
||||
" \"\"\" Display the image in the notebook. \"\"\"\n",
|
||||
" a = np.uint8(np.clip(a, 0, 255))\n",
|
||||
" f = StringIO()\n",
|
||||
" PIL.Image.fromarray(a).save(f, fmt)\n",
|
||||
" display(Image(data=f.getvalue()))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"id": "i9hkSm1IOZNR"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"model_path = '../models/bvlc_googlenet/'\n",
|
||||
"net_fn = os.path.join(model_path, 'deploy.prototxt')\n",
|
||||
"param_fn = os.path.join(model_path, 'bvlc_googlenet.caffemodel')\n",
|
||||
"\n",
|
||||
"# Patching model to be able to compute gradients.\n",
|
||||
"# Note that you can also manually add \"force_backward: true\" line to \"deploy.prototxt\".\n",
|
||||
"model = caffe.io.caffe_pb2.NetParameter()\n",
|
||||
"text_format.Merge(open(net_fn).read(), model)\n",
|
||||
"model.force_backward = True\n",
|
||||
"open('tmp.prototxt', 'w').write(str(model))\n",
|
||||
"\n",
|
||||
"net = caffe.Classifier('tmp.prototxt', param_fn,\n",
|
||||
" mean = np.float32([104.0, 116.0, 122.0]), # ImageNet mean, training set dependent\n",
|
||||
" channel_swap = (2,1,0)) # the reference model has channels in BGR order instead of RGB\n",
|
||||
"\n",
|
||||
"# A couple of utility functions for converting to and from Caffe's input image layout.\n",
|
||||
"def preprocess(net, img):\n",
|
||||
" return np.float32(np.rollaxis(img, 2)[::-1]) - net.transformer.mean['data']\n",
|
||||
"\n",
|
||||
"def deprocess(net, img):\n",
|
||||
" return np.dstack((img + net.transformer.mean['data'])[::-1])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "UeV_fJ4QOZNb"
|
||||
},
|
||||
"source": [
|
||||
"## Producing dreams"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "9udrp3efOZNd"
|
||||
},
|
||||
"source": [
|
||||
"The \"dream\" images is just a gradient ascent process that tries to maximize the **L2** norm of activations of a particular DNN layer. \n",
|
||||
"\n",
|
||||
"Here are a few simple tricks that we found useful for getting good images:\n",
|
||||
"\n",
|
||||
"* offset image by a random jitter,\n",
|
||||
"* normalize the magnitude of gradient ascent steps,\n",
|
||||
"* apply ascent across multiple scales (octaves),\n",
|
||||
"\n",
|
||||
"First, we implement a basic gradient ascent step function, applying the first two tricks:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"id": "pN43nMsHOZNg"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def objective_L2(dst):\n",
|
||||
" dst.diff[:] = dst.data \n",
|
||||
"\n",
|
||||
"def make_step(net, step_size=1.5, end='inception_4c/output', \n",
|
||||
" jitter=32, clip=True, objective=objective_L2):\n",
|
||||
" '''Basic gradient ascent step.'''\n",
|
||||
"\n",
|
||||
" src = net.blobs['data'] # input image is stored in Net's 'data' blob\n",
|
||||
" dst = net.blobs[end]\n",
|
||||
"\n",
|
||||
" ox, oy = np.random.randint(-jitter, jitter+1, 2)\n",
|
||||
" src.data[0] = np.roll(np.roll(src.data[0], ox, -1), oy, -2) # apply jitter shift\n",
|
||||
" \n",
|
||||
" net.forward(end=end)\n",
|
||||
" objective(dst) # specify the optimization objective\n",
|
||||
" net.backward(start=end)\n",
|
||||
" g = src.diff[0]\n",
|
||||
" # apply normalized ascent step to the input image\n",
|
||||
" src.data[:] += step_size/np.abs(g).mean() * g\n",
|
||||
"\n",
|
||||
" src.data[0] = np.roll(np.roll(src.data[0], -ox, -1), -oy, -2) # unshift image\n",
|
||||
" \n",
|
||||
" if clip:\n",
|
||||
" bias = net.transformer.mean['data']\n",
|
||||
" src.data[:] = np.clip(src.data, -bias, 255-bias) "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "nphEdlBgOZNk"
|
||||
},
|
||||
"source": [
|
||||
"Next we implement an ascent through different scales. We call these scales \"octaves\"."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"id": "ZpFIn8l0OZNq"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def deepdream(net, base_img, iter_n=10, octave_n=4, octave_scale=1.4, \n",
|
||||
" end='inception_4c/output', clip=True, **step_params):\n",
|
||||
" \n",
|
||||
" # prepare base images for all octaves\n",
|
||||
" octaves = [preprocess(net, base_img)]\n",
|
||||
" for i in xrange(octave_n-1):\n",
|
||||
" octaves.append(nd.zoom(octaves[-1], (1, 1.0/octave_scale,1.0/octave_scale), order=1))\n",
|
||||
" \n",
|
||||
" src = net.blobs['data']\n",
|
||||
" detail = np.zeros_like(octaves[-1]) # allocate image for network-produced details\n",
|
||||
" for octave, octave_base in enumerate(octaves[::-1]):\n",
|
||||
" h, w = octave_base.shape[-2:]\n",
|
||||
" if octave > 0:\n",
|
||||
" # upscale details from the previous octave\n",
|
||||
" h1, w1 = detail.shape[-2:]\n",
|
||||
" detail = nd.zoom(detail, (1, 1.0*h/h1,1.0*w/w1), order=1)\n",
|
||||
"\n",
|
||||
" src.reshape(1,3,h,w) # resize the network's input image size\n",
|
||||
" src.data[0] = octave_base+detail\n",
|
||||
" for i in xrange(iter_n):\n",
|
||||
" make_step(net, end=end, clip=clip, **step_params)\n",
|
||||
" \n",
|
||||
" # visualization\n",
|
||||
" vis = deprocess(net, src.data[0])\n",
|
||||
" if not clip: # adjust image contrast if clipping is disabled\n",
|
||||
" vis = vis*(255.0/np.percentile(vis, 99.98))\n",
|
||||
" showarray(vis)\n",
|
||||
" print octave, i, end, vis.shape\n",
|
||||
" clear_output(wait=True)\n",
|
||||
" \n",
|
||||
" # extract details produced on the current octave\n",
|
||||
" detail = src.data[0]-octave_base\n",
|
||||
" # returning the resulting image\n",
|
||||
" return deprocess(net, src.data[0])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "QrcdU-lmOZNx"
|
||||
},
|
||||
"source": [
|
||||
"Now we are ready to let the neural network reveal its dreams! Let's take a Dali image as a starting point:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"executionInfo": null,
|
||||
"id": "40p5AqqwOZN5",
|
||||
"outputId": "f62cde37-79e8-420a-e448-3b9b48ee1730",
|
||||
"pinned": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"img = np.float32(PIL.Image.open(BACKGROUND_IMG))\n",
|
||||
"showarray(img)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "Z9_215_GOZOL"
|
||||
},
|
||||
"source": [
|
||||
"Running the next code cell starts the detail generation process. You may see how new patterns start to form, iteration by iteration, octave by octave."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"executionInfo": null,
|
||||
"id": "HlnVnDTlOZOL",
|
||||
"outputId": "425dfc83-b474-4a69-8386-30d86361bbf6",
|
||||
"pinned": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"_=deepdream(net, img)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "Rp9kOCQTOZOQ"
|
||||
},
|
||||
"source": [
|
||||
"The complexity of the details generated depends on which layer's activations we try to maximize. Higher layers produce complex features, while lower ones enhance edges and textures, giving the image an impressionist feeling:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"executionInfo": null,
|
||||
"id": "eHOX0t93OZOR",
|
||||
"outputId": "0de0381c-4681-4619-912f-9b6a2cdec0c6",
|
||||
"pinned": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"_=deepdream(net, img, end='inception_3b/5x5_reduce')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "rkzHz9E8OZOb"
|
||||
},
|
||||
"source": [
|
||||
"We encourage readers to experiment with layer selection to see how it affects the results. Execute the next code cell to see the list of different layers. You can modify the `make_step` function to make it follow some different objective, say to select a subset of activations to maximize, or to maximize multiple layers at once. There is a huge design space to explore!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"id": "OIepVN6POZOc"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"net.blobs.keys()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "vs2uUpMCOZOe"
|
||||
},
|
||||
"source": [
|
||||
"What if we feed the `deepdream` function its own output, after applying a little zoom to it? It turns out that this leads to an endless stream of impressions of the things that the network saw during training. Some patterns fire more often than others, suggestive of basins of attraction.\n",
|
||||
"\n",
|
||||
"We will start the process from the same sky image as above, but after some iteration the original image becomes irrelevant; even random noise can be used as the starting point."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"id": "IB48CnUfOZOe"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!mkdir frames\n",
|
||||
"frame = img\n",
|
||||
"frame_i = 0"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"id": "fj0E-fKDOZOi"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"h, w = frame.shape[:2]\n",
|
||||
"s = 0.05 # scale coefficient\n",
|
||||
"\n",
|
||||
"for i in xrange(1):\n",
|
||||
" frame = deepdream(net, frame)\n",
|
||||
" PIL.Image.fromarray(np.uint8(frame)).save(\"frames/%04d.jpg\"%frame_i)\n",
|
||||
" frame = nd.affine_transform(frame, [1-s,1-s,1], [h*s/2,w*s/2,0], order=1)\n",
|
||||
" frame_i += 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"colab_type": "text",
|
||||
"id": "XzZGGME_OZOk"
|
||||
},
|
||||
"source": [
|
||||
"Be careful running the code above, it can bring you into very strange realms!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"cellView": "both",
|
||||
"colab_type": "code",
|
||||
"collapsed": false,
|
||||
"executionInfo": null,
|
||||
"id": "ZCZcz2p1OZOt",
|
||||
"outputId": "d3773436-2b5d-4e79-be9d-0f12ab839fff",
|
||||
"pinned": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"Image(filename='frames/0029.jpg')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Controlling dreams\n",
|
||||
"\n",
|
||||
"The image detail generation method described above tends to produce some patterns more often the others. One easy way to improve the generated image diversity is to tweak the optimization objective. Here we show just one of many ways to do that. Let's use one more input image. We'd call it a \"*guide*\"."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"guide = np.float32(PIL.Image.open(CONTROL_IMAGE))\n",
|
||||
"showarray(guide)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note that the neural network we use was trained on images downscaled to 224x224 size. So high resolution images might have to be downscaled, so that the network could pick up their features. The image we use here is already small enough.\n",
|
||||
"\n",
|
||||
"Now we pick some target layer and extract guide image features."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"end = 'inception_3b/output'\n",
|
||||
"h, w = guide.shape[:2]\n",
|
||||
"src, dst = net.blobs['data'], net.blobs[end]\n",
|
||||
"src.reshape(1,3,h,w)\n",
|
||||
"src.data[0] = preprocess(net, guide)\n",
|
||||
"net.forward(end=end)\n",
|
||||
"guide_features = dst.data[0].copy()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Instead of maximizing the L2-norm of current image activations, we try to maximize the dot-products between activations of current image, and their best matching correspondences from the guide image."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def objective_guide(dst):\n",
|
||||
" x = dst.data[0].copy()\n",
|
||||
" y = guide_features\n",
|
||||
" ch = x.shape[0]\n",
|
||||
" x = x.reshape(ch,-1)\n",
|
||||
" y = y.reshape(ch,-1)\n",
|
||||
" A = x.T.dot(y) # compute the matrix of dot-products with guide features\n",
|
||||
" dst.diff[0].reshape(ch,-1)[:] = y[:,A.argmax(1)] # select ones that match best\n",
|
||||
"\n",
|
||||
"_=deepdream(net, img, end=end, objective=objective_guide)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This way we can affect the style of generated images without using a different training set."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"colabVersion": "0.3.1",
|
||||
"default_view": {},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 2",
|
||||
"language": "python",
|
||||
"name": "python2"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.11"
|
||||
},
|
||||
"views": {}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
BIN
Deep_Art/deepdream/examples/flowers.jpg
Normal file
BIN
Deep_Art/deepdream/examples/flowers.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 30 KiB |
6911
Deep_Art/deepdream/examples/net_surgery.ipynb
Normal file
6911
Deep_Art/deepdream/examples/net_surgery.ipynb
Normal file
File diff suppressed because it is too large
Load diff
BIN
Deep_Art/deepdream/examples/sky1024px.jpg
Normal file
BIN
Deep_Art/deepdream/examples/sky1024px.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 64 KiB |
2159
Deep_Art/deepdream/examples/tmp.prototxt
Normal file
2159
Deep_Art/deepdream/examples/tmp.prototxt
Normal file
File diff suppressed because it is too large
Load diff
85
Numpy/nn_simple.ipynb
Normal file
85
Numpy/nn_simple.ipynb
Normal file
File diff suppressed because one or more lines are too long
36
TensorFlow/intro_linear_model.py
Normal file
36
TensorFlow/intro_linear_model.py
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
import numpy as np
|
||||
import tensorflow as tf
|
||||
|
||||
# Model linear regression y = Wx + b
|
||||
x = tf.placeholder(tf.float32, [None, 1])
|
||||
W = tf.Variable(tf.zeros([1,1]))
|
||||
b = tf.Variable(tf.zeros([1]))
|
||||
product = tf.matmul(x,W)
|
||||
y = product + b
|
||||
y_ = tf.placeholder(tf.float32, [None, 1])
|
||||
|
||||
# Cost function sum((y_-y)**2)
|
||||
cost = tf.reduce_mean(tf.square(y_-y))
|
||||
|
||||
# Training using Gradient Descent to minimize cost
|
||||
train_step = tf.train.GradientDescentOptimizer(0.0000001).minimize(cost)
|
||||
|
||||
sess = tf.Session()
|
||||
init = tf.initialize_all_variables()
|
||||
sess.run(init)
|
||||
steps = 1000
|
||||
for i in range(steps):
|
||||
|
||||
# Create fake data for y = W.x + b where W = 2, b = 0
|
||||
xs = np.array([[i]])
|
||||
ys = np.array([[2*i]])
|
||||
|
||||
# Train
|
||||
feed = { x: xs, y_: ys }
|
||||
sess.run(train_step, feed_dict=feed)
|
||||
|
||||
print("After %d iteration:" % i)
|
||||
print("W: %f" % sess.run(W))
|
||||
print("b: %f" % sess.run(b))
|
||||
|
||||
print("cost: %f" % sess.run(cost, feed_dict=feed))
|
||||
Loading…
Add table
Add a link
Reference in a new issue