管窥锥指¶

W2H2:

What is X
When to use X
How to use X
How x design and implement

Contents:

全局概览¶

编码技艺¶

编程语言 数据结构与算法 练习

Python¶

Concurrency toolset¶

Async IO & Coroutine¶

Async IO & Coroutine

Questions¶

What is Async IO
How to use asyncio
What is coroutine
How to create native coroutine
Relationship of coroutine and asyncio

Summary¶

Asynchronous IO as a language-agnostic model and a way to effect concurrency by letting coroutines indirectly communicate with each other
The specifics of Python’s new async and await keywords, used to mark and define coroutines
asyncio, the Python package that provides the API to run and manage coroutines

Async IO¶

What is Async IO¶

Async IO is a style of concurrent programming, but it is not parallelism. It’s more closely aligned with threading than with multiprocessing but is very much distinct from both of these and is a standalone member in concurrency’s bag of tricks.
Async IO is a single-threaded, single-process design: it uses cooperative multitasking.
It has been said in other words that async IO gives a feeling of concurrency despite using a single thread in a single process.
Coroutines (a central feature of async IO) can be scheduled concurrently, but they are not inherently concurrent.
Asynchronous routines are able to “pause” while waiting on their ultimate result and let other routines run in the meantime.
Asynchronous code, through the mechanism above, facilitates concurrent execution. To put it differently, asynchronous code gives the look and feel of concurrency.
Async IO takes long waiting periods in which functions would otherwise be blocking and allows other functions to run during that downtime. (A function that blocks effectively forbids others from running from the time that it starts until the time that it returns.)
Explain: 国际象棋大师1vn，饭店服务员点餐

Where Async IO fit in¶

Python `asyncio` package and `async/await`¶

native coroutine¶

rule of Async IO¶

Async IO design patterns¶

When to use Async IO¶

Example: Asynchrounous Requests¶

#!/usr/bin/env python3
# areq.py

"""Asynchronously get links embedded in multiple pages' HMTL."""

import asyncio
import logging
import re
import sys
from typing import IO
import urllib.error
import urllib.parse

import aiofiles
import aiohttp
from aiohttp import ClientSession

logging.basicConfig(
    format="%(asctime)s %(levelname)s:%(name)s: %(message)s",
    level=logging.DEBUG,
    datefmt="%H:%M:%S",
    stream=sys.stderr,
)
logger = logging.getLogger("areq")
logging.getLogger("chardet.charsetprober").disabled = True

HREF_RE = re.compile(r'href="(.*?)"')

async def fetch_html(url: str, session: ClientSession, **kwargs) -> str:
    """GET request wrapper to fetch page HTML.

    kwargs are passed to `session.request()`.
    """

    resp = await session.request(method="GET", url=url, **kwargs)
    resp.raise_for_status()
    logger.info("Got response [%s] for URL: %s", resp.status, url)
    html = await resp.text()
    return html

async def parse(url: str, session: ClientSession, **kwargs) -> set:
    """Find HREFs in the HTML of `url`."""
    found = set()
    try:
        html = await fetch_html(url=url, session=session, **kwargs)
    except (
        aiohttp.ClientError,
        aiohttp.http_exceptions.HttpProcessingError,
    ) as e:
        logger.error(
            "aiohttp exception for %s [%s]: %s",
            url,
            getattr(e, "status", None),
            getattr(e, "message", None),
        )
        return found
    except Exception as e:
        logger.exception(
            "Non-aiohttp exception occured:  %s", getattr(e, "__dict__", {})
        )
        return found
    else:
        for link in HREF_RE.findall(html):
            try:
                abslink = urllib.parse.urljoin(url, link)
            except (urllib.error.URLError, ValueError):
                logger.exception("Error parsing URL: %s", link)
                pass
            else:
                found.add(abslink)
        logger.info("Found %d links for %s", len(found), url)
        return found

async def write_one(file: IO, url: str, **kwargs) -> None:
    """Write the found HREFs from `url` to `file`."""
    res = await parse(url=url, **kwargs)
    if not res:
        return None
    async with aiofiles.open(file, "a") as f:
        for p in res:
            await f.write(f"{url}\t{p}\n")
        logger.info("Wrote results for source URL: %s", url)

async def bulk_crawl_and_write(file: IO, urls: set, **kwargs) -> None:
    """Crawl & write concurrently to `file` for multiple `urls`."""
    async with ClientSession() as session:
        tasks = []
        for url in urls:
            tasks.append(
                write_one(file=file, url=url, session=session, **kwargs)
            )
        await asyncio.gather(*tasks)

if __name__ == "__main__":
    import pathlib
    import sys

    assert sys.version_info >= (3, 7), "Script requires Python 3.7+."
    here = pathlib.Path(__file__).parent

    with open(here.joinpath("urls.txt")) as infile:
        urls = set(map(str.strip, infile))

    outpath = here.joinpath("foundurls.txt")
    with open(outpath, "w") as outfile:
        outfile.write("source_url\tparsed_url\n")

    asyncio.run(bulk_crawl_and_write(file=outpath, urls=urls))

个人更倾向于 reStructuredText 和 Google 风格

reStructuredText 风格（PyCharm 默认风格）

"""
This is a reST style.

:param param1: this is a first param
:param param2: this is a second param
:returns: this is a description of what is returned
:raises keyError: raises an exception
"""

Google 风格

"""
This is an example of Google style.

Args:
    param1: This is the first param.
    param2: This is a second param.

Returns:
    This is a description of what is returned.

Raises:
    KeyError: Raises an exception.
"""

Numpydoc 风格

"""
My numpydoc description of a kind
of very exhautive numpydoc format docstring.

Parameters
----------
first : array_like
    the 1st param name `first`
second :
    the 2nd param
third : {'value', 'other'}, optional
    the 3rd param, by default 'value'

Returns
-------
string
    a value in a string

Raises
------
KeyError
    when a key error
OtherError
    when an other error
"""

Everything about Rregular Expression¶

Everything about Rregular Expression

What is regex¶

Regex solve this problems:

Q: Does this string match the pattern?

A: match -- find something from the beginning of the string, and return it

Q: Is there a match for the pattern anywhere in this string?

A: search -- find something anywhere in the string, and return it
A picture is worth a thousand words

regx-picture

Methods and Attribute¶

Method/Attributes	Purpose
`match()`	Determine if the RE matches at the beginning of the string.
`search()`	Scan through a string, looking for any location where this RE matches.
`findall()`	Find all substrings where the RE matches, and returns them as a list.
`finditer()`	Find all substrings where the RE matches, and returns them as an iterator.

Method/Attributes	Purpose
`group()`	Return the string matched by the RE
`start()`	Return the starting position of the match
`end()`	Return the ending position of the match
`span()`	Return a tuple containing the (start, end) positions of the match

Method/Attributes	Purpose
`split()`	Split the string into a list, splitting it wherever the RE matches
`sub()`	Find all substrings where the RE matches, and replace them with a different string
`subn()`	Does the same thing as sub(), but returns the new string and the number of replacements

Example¶

import re
m = re.match(r'\d+', '123abc')
if m:
    print('yep match it')
else:
    print('Ops No match')

s = re.search('r\w+@\w+', 'you@me')

# Compile pattern first
pattern = re.compile(r'[\w-@]+')
mp = pattern.match('we@me')
sp = pattern.search('you')

def is_valid_email(string):
    email_pattern = re.compile(r'[^@]+@[^@]+\.[^@]+')
    return True if email_pattern.match(string) else False

def test_is_valid_email():
    assert is_valid_email('abc-123@gmail.com')
    assert not is_valid_email('abc@')

# Greedy vs. Non-Greedy
>>> re.match(r'<.*>', '<html>abc</html><a>happy</a>').group()  # greedy
>>> '<html>abc</html><a>happy</a>'
>>> re.match(r'<.*?>', '<html>abc</html><a>happy</a>').group()  # non-gready
>>> '<html>'

References¶

Read code¶

Read code

收集代码世界里的吉光片羽

Property demo¶

import math

class Square(object):
    """A square with two properties: a writable area and a read-only perimeter.

    To use:
    >>> sq = Square(3)
    >>> sq.area
    9
    >>> sq.perimeter
    12
    >>> sq.area = 16
    >>> sq.side
    4
    >>> sq.perimeter
    16
    """

    def __init__(self, side):
        self.side = side

    @property
    def area(self):
        """Gets or sets the area of the square."""
        return self._get_area()

    @area.setter
    def area(self, area):
        return self._set_area(area)

    def _get_area(self):
        """Indirect accessor to calculate the 'area' property."""
        return self.side ** 2

    def _set_area(self, area):
        """Indirect setter to set the 'area' property."""
        self.side = math.sqrt(area)

    @property
    def perimeter(self):
        return self.side * 4

`Sentry` code snippet¶

from __future__ import absolute_import

from datetime import datetime

from sentry.integrations.github.utils import get_jwt
from sentry.integrations.client import ApiClient

class GitHubClientMixin(ApiClient):
    allow_redirects = True

    base_url = 'https://api.github.com'

    def get_jwt(self):
        return get_jwt()

    def get_last_commits(self, repo, end_sha):
        # return api request that fetches last ~30 commits
        # see https://developer.github.com/v3/repos/commits/#list-commits-on-a-repository
        # using end_sha as parameter
        return self.get(
            u'/repos/{}/commits'.format(
                repo,
            ),
            params={'sha': end_sha},
        )

    def compare_commits(self, repo, start_sha, end_sha):
        # see https://developer.github.com/v3/repos/commits/#compare-two-commits
        # where start sha is oldest and end is most recent
        return self.get(u'/repos/{}/compare/{}...{}'.format(
            repo,
            start_sha,
            end_sha,
        ))

    def get_pr_commits(self, repo, num):
        # see https://developer.github.com/v3/pulls/#list-commits-on-a-pull-request
        # Max: 250 Commits
        return self.get(u'/repos/{}/pulls/{}/commits'.format(
            repo,
            num
        ))

    def repo_hooks(self, repo):
        return self.get(u'/repos/{}/hooks'.format(repo))

    def get_commits(self, repo):
        return self.get(u'/repos/{}/commits'.format(repo))

    def get_repo(self, repo):
        return self.get(u'/repos/{}'.format(repo))

    def get_repositories(self):
        repositories = self.get(
            '/installation/repositories',
            params={'per_page': 100},
        )
        return repositories['repositories']

    def search_repositories(self, query):
        return self.get(
            '/search/repositories',
            params={'q': query},
        )

    def get_assignees(self, repo):
        return self.get(u'/repos/{}/assignees'.format(repo))

    def get_issues(self, repo):
        return self.get(u'/repos/{}/issues'.format(repo))

    def search_issues(self, query):
        return self.get(
            '/search/issues',
            params={'q': query},
        )

    def get_issue(self, repo, number):
        return self.get(u'/repos/{}/issues/{}'.format(repo, number))

    def create_issue(self, repo, data):
        endpoint = u'/repos/{}/issues'.format(repo)
        return self.post(endpoint, data=data)

    def create_comment(self, repo, issue_id, data):
        endpoint = u'/repos/{}/issues/{}/comments'.format(repo, issue_id)
        return self.post(endpoint, data=data)

    def get_user(self, gh_username):
        return self.get(u'/users/{}'.format(gh_username))

    def request(self, method, path, headers=None, data=None, params=None):
        if headers is None:
            headers = {
                'Authorization': 'token %s' % self.get_token(),
                # TODO(jess): remove this whenever it's out of preview
                'Accept': 'application/vnd.github.machine-man-preview+json',
            }
        return self._request(method, path, headers=headers, data=data, params=params)

    def get_token(self):
        """
        Get token retrieves the active access token from the integration model.
        Should the token have expried, a new token will be generated and
        automatically presisted into the integration.
        """
        token = self.integration.metadata.get('access_token')
        expires_at = self.integration.metadata.get('expires_at')

        if expires_at is not None:
            expires_at = datetime.strptime(expires_at, '%Y-%m-%dT%H:%M:%S')

        if not token or expires_at < datetime.utcnow():
            res = self.create_token()
            token = res['token']
            expires_at = datetime.strptime(
                res['expires_at'],
                '%Y-%m-%dT%H:%M:%SZ',
            )

            self.integration.metadata.update({
                'access_token': token,
                'expires_at': expires_at.isoformat(),
            })
            self.integration.save()

        return token

    def create_token(self):
        return self.post(
            u'/installations/{}/access_tokens'.format(
                self.integration.external_id,
            ),
            headers={
                'Authorization': 'Bearer %s' % self.get_jwt(),
                # TODO(jess): remove this whenever it's out of preview
                'Accept': 'application/vnd.github.machine-man-preview+json',
            },
        )

class GitHubAppsClient(GitHubClientMixin):

    def __init__(self, integration):
        self.integration = integration
        super(GitHubAppsClient, self).__init__()

`Django` auth user model¶

胖 model 瘦 control

class UserManager(models.Manager):

    @classmethod
    def normalize_email(cls, email):
        """
        Normalize the address by lowercasing the domain part of the email
        address.
        """
        email = email or ''
        try:
            email_name, domain_part = email.strip().rsplit('@', 1)
        except ValueError:
            pass
        else:
            email = '@'.join([email_name, domain_part.lower()])
        return email

    def create_user(self, username, email=None, password=None):
        """
        Creates and saves a User with the given username, email and password.
        """
        now = timezone.now()
        if not username:
            raise ValueError('The given username must be set')
        email = UserManager.normalize_email(email)
        user = self.model(username=username, email=email,
                          is_staff=False, is_active=True, is_superuser=False,
                          last_login=now, date_joined=now)

        user.set_password(password)
        user.save(using=self._db)
        return user

    def create_superuser(self, username, email, password):
        u = self.create_user(username, email, password)
        u.is_staff = True
        u.is_active = True
        u.is_superuser = True
        u.save(using=self._db)
        return u

    def make_random_password(self, length=10,
                             allowed_chars='abcdefghjkmnpqrstuvwxyz'
                                           'ABCDEFGHJKLMNPQRSTUVWXYZ'
                                           '23456789'):
        """
        Generates a random password with the given length and given
        allowed_chars. Note that the default value of allowed_chars does not
        have "I" or "O" or letters and digits that look similar -- just to
        avoid confusion.
        """
        return get_random_string(length, allowed_chars)

    def get_by_natural_key(self, username):
        return self.get(username=username)

class User(models.Model):
    """
    Users within the Django authentication system are represented by this
    model.
    Username and password are required. Other fields are optional.
    """
    username = models.CharField(_('username'), max_length=30, unique=True,
        help_text=_('Required. 30 characters or fewer. Letters, numbers and '
                    '@/./+/-/_ characters'))
    first_name = models.CharField(_('first name'), max_length=30, blank=True)
    last_name = models.CharField(_('last name'), max_length=30, blank=True)
    email = models.EmailField(_('e-mail address'), blank=True)
    password = models.CharField(_('password'), max_length=128)
    is_staff = models.BooleanField(_('staff status'), default=False,
        help_text=_('Designates whether the user can log into this admin '
                    'site.'))
    is_active = models.BooleanField(_('active'), default=True,
        help_text=_('Designates whether this user should be treated as '
                    'active. Unselect this instead of deleting accounts.'))
    is_superuser = models.BooleanField(_('superuser status'), default=False,
        help_text=_('Designates that this user has all permissions without '
                    'explicitly assigning them.'))
    last_login = models.DateTimeField(_('last login'), default=timezone.now)
    date_joined = models.DateTimeField(_('date joined'), default=timezone.now)
    groups = models.ManyToManyField(Group, verbose_name=_('groups'),
        blank=True, help_text=_('The groups this user belongs to. A user will '
                                'get all permissions granted to each of '
                                'his/her group.'))
    user_permissions = models.ManyToManyField(Permission,
        verbose_name=_('user permissions'), blank=True,
        help_text='Specific permissions for this user.')
    objects = UserManager()

    class Meta:
        verbose_name = _('user')
        verbose_name_plural = _('users')

    def __unicode__(self):
        return self.username

    def natural_key(self):
        return (self.username,)

    def get_absolute_url(self):
        return "/users/%s/" % urllib.quote(smart_str(self.username))

    def is_anonymous(self):
        """
        Always returns False. This is a way of comparing User objects to
        anonymous users.
        """
        return False

    def is_authenticated(self):
        """
        Always return True. This is a way to tell if the user has been
        authenticated in templates.
        """
        return True

    def get_full_name(self):
        """
        Returns the first_name plus the last_name, with a space in between.
        """
        full_name = u'%s %s' % (self.first_name, self.last_name)
        return full_name.strip()

    def set_password(self, raw_password):
        self.password = make_password(raw_password)

    def check_password(self, raw_password):
        """
        Returns a boolean of whether the raw_password was correct. Handles
        hashing formats behind the scenes.
        """
        def setter(raw_password):
            self.set_password(raw_password)
            self.save()
        return check_password(raw_password, self.password, setter)

    def set_unusable_password(self):
        # Sets a value that will never be a valid hash
        self.password = make_password(None)

    def has_usable_password(self):
        return is_password_usable(self.password)

    def get_group_permissions(self, obj=None):
        """
        Returns a list of permission strings that this user has through his/her
        groups. This method queries all available auth backends. If an object
        is passed in, only permissions matching this object are returned.
        """
        permissions = set()
        for backend in auth.get_backends():
            if hasattr(backend, "get_group_permissions"):
                if obj is not None:
                    permissions.update(backend.get_group_permissions(self,
                                                                     obj))
                else:
                    permissions.update(backend.get_group_permissions(self))
        return permissions

    def get_all_permissions(self, obj=None):
        return _user_get_all_permissions(self, obj)

    def has_perm(self, perm, obj=None):
        """
        Returns True if the user has the specified permission. This method
        queries all available auth backends, but returns immediately if any
        backend returns True. Thus, a user who has permission from a single
        auth backend is assumed to have permission in general. If an object is
        provided, permissions for this specific object are checked.
        """

        # Active superusers have all permissions.
        if self.is_active and self.is_superuser:
            return True

        # Otherwise we need to check the backends.
        return _user_has_perm(self, perm, obj)

    def has_perms(self, perm_list, obj=None):
        """
        Returns True if the user has each of the specified permissions. If
        object is passed, it checks if the user has all required perms for this
        object.
        """
        for perm in perm_list:
            if not self.has_perm(perm, obj):
                return False
        return True

    def has_module_perms(self, app_label):
        """
        Returns True if the user has any permissions in the given app label.
        Uses pretty much the same logic as has_perm, above.
        """
        # Active superusers have all permissions.
        if self.is_active and self.is_superuser:
            return True

        return _user_has_module_perms(self, app_label)

    def email_user(self, subject, message, from_email=None):
        """
        Sends an email to this User.
        """
        send_mail(subject, message, from_email, [self.email])

    def get_profile(self):
        """
        Returns site-specific profile for this user. Raises
        SiteProfileNotAvailable if this site does not allow profiles.
        """
        if not hasattr(self, '_profile_cache'):
            from django.conf import settings
            if not getattr(settings, 'AUTH_PROFILE_MODULE', False):
                raise SiteProfileNotAvailable(
                    'You need to set AUTH_PROFILE_MODULE in your project '
                    'settings')
            try:
                app_label, model_name = settings.AUTH_PROFILE_MODULE.split('.')
            except ValueError:
                raise SiteProfileNotAvailable(
                    'app_label and model_name should be separated by a dot in '
                    'the AUTH_PROFILE_MODULE setting')
            try:
                model = models.get_model(app_label, model_name)
                if model is None:
                    raise SiteProfileNotAvailable(
                        'Unable to load the profile model, check '
                        'AUTH_PROFILE_MODULE in your project settings')
                self._profile_cache = model._default_manager.using(
                                   self._state.db).get(user__id__exact=self.id)
                self._profile_cache.user = self
            except (ImportError, ImproperlyConfigured):
                raise SiteProfileNotAvailable
        return self._profile_cache

`Requests` code show¶

from .compat import OrderedDict, Mapping, MutableMapping

class CaseInsensitiveDict(MutableMapping):
    """A case-insensitive ``dict``-like object.

    Implements all methods and operations of
    ``MutableMapping`` as well as dict's ``copy``. Also
    provides ``lower_items``.

    All keys are expected to be strings. The structure remembers the
    case of the last key to be set, and ``iter(instance)``,
    ``keys()``, ``items()``, ``iterkeys()``, and ``iteritems()``
    will contain case-sensitive keys. However, querying and contains
    testing is case insensitive::

        cid = CaseInsensitiveDict()
        cid['Accept'] = 'application/json'
        cid['aCCEPT'] == 'application/json'  # True
        list(cid) == ['Accept']  # True

    For example, ``headers['content-encoding']`` will return the
    value of a ``'Content-Encoding'`` response header, regardless
    of how the header name was originally stored.

    If the constructor, ``.update``, or equality comparison
    operations are given keys that have equal ``.lower()``s, the
    behavior is undefined.
    """

    def __init__(self, data=None, **kwargs):
        self._store = OrderedDict()
        if data is None:
            data = {}
        self.update(data, **kwargs)

    def __setitem__(self, key, value):
        # Use the lowercased key for lookups, but store the actual
        # key alongside the value.
        self._store[key.lower()] = (key, value)

    def __getitem__(self, key):
        return self._store[key.lower()][1]

    def __delitem__(self, key):
        del self._store[key.lower()]

    def __iter__(self):
        return (casedkey for casedkey, mappedvalue in self._store.values())

    def __len__(self):
        return len(self._store)

    def lower_items(self):
        """Like iteritems(), but with all lowercase keys."""
        return (
            (lowerkey, keyval[1])
            for (lowerkey, keyval)
            in self._store.items()
        )

    def __eq__(self, other):
        if isinstance(other, Mapping):
            other = CaseInsensitiveDict(other)
        else:
            return NotImplemented
        # Compare insensitively
        return dict(self.lower_items()) == dict(other.lower_items())

    # Copy is required
    def copy(self):
        return CaseInsensitiveDict(self._store.values())

    def __repr__(self):
        return str(dict(self.items()))

The Zen of Python¶

Beautiful is better than ugly.

Explicit is better than implicit.

Simple is better than complex.

Complex is better than complicated.

Flat is better than nested.

Sparse is better than dense.

Readability counts.

Special cases aren't special enough to break the rules.

Although practicality beats purity.

Errors should never pass silently.

Unless explicitly silenced.

In the face of ambiguity, refuse the temptation to guess.

There should be one-- and preferably only one --obvious way to do it.

Although that way may not be obvious at first unless you're Dutch.

Now is better than never.

Although never is often better than *right* now.

If the implementation is hard to explain, it's a bad idea.

If the implementation is easy to explain, it may be a good idea.

Namespaces are one honking great idea -- let's do more of those

by Tim Peters

Go¶

The Go Programming language¶

Go 程序设计语言笔记

前言¶

Go 语言及其配套工具集意在使得编译和执行都能够富有表达能力和效率, 而且使得程序员能够写出可靠、健壮的程序
Communicating Sequential Process(CSP)中程序就是一组无共享状态进程的并行组合，进程间通信和同步采用信道完成
在不断要求增加系统功能和选项以及配置，还要求快速发布的压力之下，简单性往往被忽视了--尽管长期来看简单性才是好软件的不二法门
要实现简单性，就要在项目的一开始将思想浓缩至本质，并在项目的整个生命周期多保持一些自律，认识到那些变化是好的，哪些变化是坏的或致命的。
KISS

安装环境搭建指南¶

本书示例代码¶

go get gopl.io

Rust¶

Rust programming language¶

简介¶

Rust 是一种系统编程语言。它有着惊人的运行速度，能够防止段错误，并保证线程安全。

Rust is a programming language that’s focused on safety, speed, and concurrency. Its design lets you create programs that have the performance and control of a low-level language, but with the powerful abstractions of a high-level language. These properties make Rust suitable for programmers who have experience in languages like C and are looking for a safer alternative, as well as those from languages like Python who are looking for ways to write code that performs better without sacrificing expressiveness.

特点¶

零开销抽象
转义语义
保证内存安全
线程无数据竞争
基于 trait 的泛型
模式匹配
类型推断
极小运行时
高效 C 绑定

实例¶

fn main() {
    let greetings = ["Hello", "Hola", "Bonjour",
                     "Ciao", "こんにちは", "안녕하세요",
                     "Cześć", "Olá", "Здравствуйте",
                     "Chào bạn", "您好", "Hallo",
                     "Hej", "Ahoj", "سلام"];

    for (num, greeting) in greetings.iter().enumerate() {
        print!("{} : ", greeting);
        match num {
            0 =>  println!("This code is editable and runnable!"),
            1 =>  println!("¡Este código es editable y ejecutable!"),
            2 =>  println!("Ce code est modifiable et exécutable !"),
            3 =>  println!("Questo codice è modificabile ed eseguibile!"),
            4 =>  println!("このコードは編集して実行出来ます！"),
            5 =>  println!("여기에서 코드를 수정하고 실행할 수 있습니다!"),
            6 =>  println!("Ten kod można edytować oraz uruchomić!"),
            7 =>  println!("Este código é editável e executável!"),
            8 =>  println!("Этот код можно отредактировать и запустить!"),
            9 =>  println!("Bạn có thể edit và run code trực tiếp!"),
            10 => println!("这段代码是可以编辑并且能够运行的！"),
            11 => println!("Dieser Code kann bearbeitet und ausgeführt werden!"),
            12 => println!("Den här koden kan redigeras och köras!"),
            13 => println!("Tento kód můžete upravit a spustit"),
            14 => println!("این کد قابلیت ویرایش و اجرا دارد!"),
            _ =>  {},
        }
    }
}

C++¶

系统设计¶

高可用高可靠可扩展系统设计研究

鸟与青蛙 Birds's-eyes view Frog dive into detail

飞鸟¶

开阔视野

概览¶

概览

系统设计指南¶

路线图¶

开发相关技术学习路线参考

后端开发路线图¶

backend-road-map

DevOps 路线图¶

devops-roadmap

Kubernets vs. Docker¶

Read list of system design¶

System design primer¶

The System Design Primer

Awesome scalability¶

Awesome Scalability, Availability, and Stability Back-end Design Patterns

青蛙¶

深入组件细节, 越系统的学习某样东西，他就会变得越来越有趣。

程序构建 ¶

程序构建

一般源代码提供的程序安装需要通过配置、编译、安装三个步骤；

配置做的工作主要是检查当前环境是否满足要安装软件的依赖关系,以及设置程序安装所需要的初始化信息，比如安装路径，需要安装哪些组件；配置完成，会生成makefile文件供第二步make使用；
编译是对源文件进行编译链接生成可执行程序；
安装做的工作就简单多了，就是将生成的可执行文件拷贝到配置时设置的初始路径下；

安装 ¶

安装做的工作就简单多了，就是将生成的可执行文件拷贝到配置时设置的初始路径下:

$make install

其实 install 就是makefile中的一个规则，打开makefile文件后可以查看程序安装的所做的工作；

总结 ¶

configure make install g++

[1]	陈皓跟我一起写Makefile http://scc.qibebt.cas.cn/docs/linux/base/%B8%FA%CE%D2%D2%BB%C6%F0%D0%B4Makefile-%B3%C2%F0%A9.pdfgtg

深入理解 Redis¶

《Redis 实战》-> 《Redis开发与运维》->《Redis 设计与实现》

Redis 使用¶

Redis 维护¶

Redis 原理¶

SQL Indexing and Tuning ¶

Contents

SQL Indexing and Tuning

Preface: why is indexing is a development task ¶

Separate what and how:

An SQL statement is a straight description what is needed without instructions as to how to get it done.

Developer must know a little bit about the database how run SQL to prevent performance problems

The most important information for indexing is how the application queries the data.

Anatomy of an Index (索引解剖)¶

注解

It is about the fundamental structure of an index

What is index

An index is a distinct structure in the database that is built using the create index statement. It requires its own disk space and holds a copy of the indexed table data. That means that an index is pure redundancy. Creating an index does not change the table data; it just creates a new data structure that refers to the table. A database index is, after all, very much like the index at the end of a book: it occupies its own space, it is highly redundant, and it refers to the actual information stored in a different place.

Why use index

An index makes the query fast

SQL database must process insert, delete and update statements immediately, keeping the index order without moving large amounts of data.

The database combines two data structures to meet the challenge: a doubly linked list and a search tree. These two structures explain most of the database's performance characteristics.

The Index Leaf Nodes¶

How to structure index¶

注解

有些数学家是鸟，其他的则是青蛙。鸟翱翔在高高的天空，俯瞰延伸至遥远地平线的广袤的数学远景。他们喜欢那些统一我们思想、并将不同领域的诸多问题整合起来的概念。青蛙生活在天空下的泥地里，只看到周围生长的花儿。他们乐于探索特定问题的细节，一次只解决一个问题。我碰巧是一只青蛙，但我的许多最好朋友都是鸟。

数学既需要鸟也需要青蛙。数学丰富又美丽，因为鸟赋予它辽阔壮观的远景，青蛙则澄清了它错综复杂的细节。数学既是伟大的艺术，也是重要的科学，因为它将普遍的概念与深邃的结构融合在一起。如果声称鸟比青蛙更好，因为它们看得更遥远，或者青蛙比鸟更好，因为它们更加深刻，那么这些都是愚蠢的见解。数学的世界既辽阔又深刻，我们需要鸟们和青蛙们协同努力来探索。

From 《鸟与青蛙》by 弗里曼•戴森

基础知识¶

计算机基础知识

“求木之长者，必固其根本；欲流之远者，必浚其泉源” 唐魏徵

基础知识¶

计算机原理¶

操作系统¶

网络协议¶

工具利器¶

开发工具，利器

工具利器¶

工具利器

效率工具¶

开发工具¶

文档工具¶

学习资料¶

MIT Hacker tools

读书笔记¶

承认无知-求取新知-应用实践

书单 ¶

Contents

书单
- 在读
- 想读
- 已读

在读 ¶

500 lines or less
霍比特人

想读 ¶

技术类¶

代码大全
编程精粹
CSAPP
APUE
AOSA

文学类¶

科学类¶

财经类¶

人文类¶

已读 ¶

AOSA¶

The Architecture of Open Source Applications [1]

500 Lines or less¶

CI¶

A Continuous Integration System

Crawler¶

A Web Crawler With asyncio Coroutines crawler, code

Introduction¶

500 Lines or Less focuses on the design decisions that programmers make in the small when they are building something new.

注解

Before reading each chapter, we encourage you to first think about how you might solve the problem.
What design considerations or constraints do you think the author is going to consider important?
What abstractions do you expect to see?
How do you think the problem is going to be decomposed?
Then, when reading the chapter, try to identify what surprised you.
It is our hope that you will learn more by doing this than by simply reading through each chapter from beginning to end.

地址 500lines

代码 500code

原文 introduce

阅读笔记¶

零散的文章阅读笔记不限类型，技术人文艺术科学文学

Learning skill and mindset ¶

Contents

Learning skill and mindset
- Blieve you can change

Blieve you can change ¶

注解

Two kind of mindset: Growth mindset vs Fixed mindset. Try to change you mindset to Growth mindset

Growth mindset ¶

What

That everything came through effort and that the world was full of interesting challenges that could help you learn and grow.

Example

The successful kids didn’t just live with failure, they loved it! When the going got tough, they didn’t start blaming themselves; they licked their lips and said “I love a challenge.” They’d say stuff like “The harder it gets the harder I need to try.”

That’s why they were so thrilled by the harder puzzles — the easier ones weren’t any sort of challenge, there was nothing you could learn from them. But the really tough ones? Those were fascinating — a new skill to develop, a new problem to conquer. In later experiments, kids even asked to take puzzles home so they could work on them some more.

“I think intelligence is something you have to work for…it isn’t just given to you… Most kids, if they’re not sure of an answer, will not raise their hand… But what I usually do is raise my hand, because if I’m wrong, then my mistake will be corrected. Or I will raise my hand and say… ‘I don’t get this. Can you help me?’ Just by doing that I’m increasing my intelligence.”

Fixed mindset ¶

What

The belief that your abilities are fixed and that the world is just a series of tests that show you how good you are.

Example

Very quickly, the helpless kids started blaming themselves: “I’m getting confused,” one said; “I never did have a good rememory,” another explained. But the puzzles kept coming — and they kept getting harder. “This isn’t fun anymore,” the kids cried.

Always thought “human qualities were carved in stone. You were smart or you weren’t, and failure meant you weren’t.” That was why the helpless kids couldn’t take it when they started failing. It just reminded them they sucked (they easily got confused, they had “a bad rememory”). Of course it wasn’t fun anymore — why would it be fun to get constantly reminded you’re a failure? No wonder they tried to change the subject.

Growth mindset vs Fixed mindset ¶

In the fixed mindset, success comes from proving how great you are. Effort is a bad thing — if you have to try hard and ask questions, you obviously can’t be very good. When you find something you can do well, you want to do it over and over, to show how good you are at it.
In the growth mindset, success comes from growing. Effort is what it’s all about — it’s what makes you grow. When you get good at something, you put it aside and look for something harder so that you can keep growing.
Fixed mindset people feel smart when they don’t make mistakes.
Growth mindset people feel smart when they struggle with something for a long time and then finally figure it out.
Fixies try to blame the world when things go bad.
Growthers look to see what they can change about themselves.
Fixies are afraid to try hard because if they fail, it means they’re a failure.
Growthers are afraid of not trying.
In relationships, growth-mindset people looked for partners who would push them to be better, fixies just wanted someone who would put them on a pedestal (and got into terrible fights when they hit problems). Even in sports, growther athletes got better and better through constant practice, while fixies blamed their atrophying skills on everyone around them.

How to start Growth mindset ¶

The first step to getting better is believing you can get better.
Try to approach the problem as a chance to grow, rather than a test of our abilities. It’s no longer scary, it’s just another project to work on.
For example, I used to think I was introverted. Everyone had always told me that you were either an extroverted person or an introverted person. From a young age, I was quite shy and bookish, so it seemed obvious: I was an introvert. But as I’ve grown, I’ve found that’s hardly the end of the story. I’ve started to get good at leading a conversation or cracking people up with a joke. I like telling stories at a party a story or buzzing about a room saying ‘hi’ to people. I get a rush from it! Sure, I’m still not the most party-oriented person I know, but I no longer think we fit into any neat introversion/extroversion buckets.

深入理解 Redis¶

《Redis 实战》-> 《Redis开发与运维》->《Redis 设计与实现》

Redis 实战¶

Redis 命令¶

Redis 命令
- 字符串
- 列表
- 集合
- 有序集合
- 散列
- 发布与订阅
- 其他命令

管窥锥指¶

全局概览¶

编码技艺¶

Python¶

Concurrency toolset¶

Async IO & Coroutine¶

Questions¶

Summary¶

Async IO¶

What is Async IO¶

Where Async IO fit in¶

Python asyncio package and async/await¶

native coroutine¶

rule of Async IO¶

Async IO design patterns¶

When to use Async IO¶

Example: Asynchrounous Requests¶

Reference¶

Code style guide¶

docsting format 规范¶

Everything about Rregular Expression¶

What is regex¶

Methods and Attribute¶

Example¶

References¶

Read code¶

Property demo¶

Sentry code snippet¶

Django auth user model¶

Requests code show¶

The Zen of Python¶

Go¶

The Go Programming language¶

前言¶

安装环境搭建指南¶

本书示例代码¶

Rust¶

Rust programming language¶

简介¶

特点¶

实例¶

C¶

C++¶

系统设计¶

飞鸟¶

概览¶

系统设计指南¶

路线图¶

后端开发路线图¶

DevOps 路线图¶

Kubernets vs. Docker¶

Read list of system design¶

System design primer¶

Awesome scalability¶

青蛙¶

深入理解 Redis¶

Redis 使用¶

Redis 维护¶

Redis 原理¶

The Index Leaf Nodes¶

基础知识¶

基础知识¶

计算机原理¶

操作系统¶

网络协议¶

工具利器¶

工具利器¶

效率工具¶

开发工具¶

文档工具¶

学习资料¶

读书笔记¶

技术类¶

文学类¶

科学类¶

财经类¶

人文类¶

AOSA¶

500 Lines or less¶

CI¶

Python `asyncio` package and `async/await`¶

`Sentry` code snippet¶

`Django` auth user model¶

`Requests` code show¶