Matches in Nanopublications for { ?s ?p ?o <https://w3id.org/np/RAKmi8CK-UdWMP9L6Q9YymxVSFsIlM1cP-6WDc4T4ocN0#assertion>. }
Showing items 1 to 15 of
15
with 100 items per page.
- assertion comment " Evolver, model merging in a genetic algorithm Improves on current merging techniques (my beloved TIES 🫣 ) Train diverse models Merge regularly or take diff between two models Update some parameters Keep if good Repeat https://arxiv.org/abs/2406.12208 @jingli9111 @banting_liu @576gsk https://twitter.com/LChoshen/status/1803410440535326786/photo/1 Merging is aimed at taking many models and getting one that generalizes better, there are various methods for it, read more e.g. on TIES https://x.com/prateeky2806/status/1665759148380758022 Genetic algorithms evolve models, in steps: Create mutations (here new m = m_old + a(m_1-m_2)) m are models a some constant Crossover, take some of the mutation and apply it, for each parameter randomly keep m_old or update to m_new Survive, keep only the best performing on val By sometimes merging and sometimes evolving (and dev sets) they improve over all current methods https://twitter.com/LChoshen/status/1803410445635653960/photo/1 In some sense, this can be seen as a better search in the region between the merged models, which we know is not equally good but all better than the edges https://x.com/LChoshen/status/1729488495515713672 https://twitter.com/LChoshen/status/1803410447246250483/photo/1 " assertion.
- assertion reviews 2406.12208 assertion.
- assertion discusses 1729488495515713672 assertion.
- assertion discusses 1665759148380758022 assertion.
- assertion creator RAoSadUw99CeqDlR2400018nqTzR_38fT86OrTzk16Vts assertion.
- assertion endorses 1729488495515713672 assertion.
- assertion keywords "TIES" assertion.
- assertion keywords "evolver" assertion.
- assertion keywords "genetic\_algorithms" assertion.
- assertion keywords "knowledge\_fusion" assertion.
- assertion keywords "model\_merging" assertion.
- 2406.12208 hasZoteroItemType "preprint" assertion.
- 1729488495515713672 hasZoteroItemType "forumPost" assertion.
- 1665759148380758022 hasZoteroItemType "forumPost" assertion.
- assertion summarizes 2406.12208 assertion.