Python 与 Go 开发笔记合集
本文把若干零散的排查与操作记录合并在一起，每条都是当时环境下的实测结果；环境变化后命令可能失效，以官方文档为准。
一起来学 Go –（4）常用函数
1. main 和 init 函数
一个 package 里面可以写多个 init 函数，但必须仅包含一个 main 函数。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
| package main
import (
"fmt"
)
func init() {
fmt.Println("init 1")
}
func init() {
fmt.Println("init 2")
}
func main() {
fmt.Println("main")
}
// init 1
// init 2
// main
|
2. defer 函数
Go 不会立即执行 defer 修饰的代码，而是进行标记，在程序退出之前执行。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| package main
import (
"fmt"
)
func main() {
defer func() {
fmt.Println("before exit, in defer")
}()
fmt.Println("I am in main")
}
// I am in main
// before exit, in defer
|
3. panic 和 recover 函数
Go 中的异常处理：抛出一个 panic ，然后在 defer 中使用 recover 捕获异常，并将 panic 抛向上一层。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
| package main
import (
"fmt"
)
func main() {
defer fmt.Println("I am in main")
defer func() {
if err := recover(); err != nil {
fmt.Println(err)
// do something
}
}()
panic("error message")
}
// error message
// I am in main
|
4. new 和 make 函数
原型：
示例：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| package main
import (
"fmt"
)
func main() {
tmp := new(int)
fmt.Printf("tmp --> %#v \n", tmp)
fmt.Printf("tmp point to --> %#v \n ", *tmp)
}
// tmp --> (*int)(0x40e020)
// tmp point to --> 0
|
new 用来分配内存，参数是类型，返回值是一个指向分配零值的指针。
原型：
1
| func make(Type, size IntegerType) Type
|
示例：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
| package main
import (
"fmt"
)
func main() {
var s1 []int
fmt.Printf("s1 is --> %#v \n", s1)
s2 := make([]int, 3)
fmt.Printf("s2 is --> %#v \n", s2)
}
// s1 is --> []int(nil)
// s2 is --> []int{0, 0, 0}
|
make 用来为 slice，map 或 chan 类型分配内存、并初始化一个对象，参数是类型。但 make 返回类型的引用而不是指针。
区别：new 仅将内存置零，而 make 会对 slice、map、chan 引用的数据进行初始化。
Django 开发中常用命令
创建 Django 工程
1
| django-admin startproject your_project_name
|
创建应用
1
2
3
| django-admin.py startapp your_app_name
# 或者
python manage.py startapp your_app_name
|
python manage.py 和 django-admin 的功能基本一样。不同的是 python manage.py 还设置了 DJANGO_SETTINGS_MODULE 环境变量、将项目的路径加入了 sys.path 中。建议除了创建项目使用 django-admin，其他情况使用 python manage.py。
初始化数据
1
| python manage.py migrate
|
创建缓存表
1
2
| python manage.py createcachetable [cache_table_name]
# 默认表名 django_cache
|
清除全部数据
1
| python manage.py flush --noinput
|
在指定端口启动服务
1
| python manage.py runserver 0.0.0.0:8000
|
启动 celery 的后台任务
1
| python manage.py celery worker --settings=settings -l info -c 4 --autoreload
|
启动 celery 的周期任务
1
| python manage.py celery beat
|
安装项目的依赖包
1
| pip install -r requirements.txt
|
关闭全部 Python 进程
1
2
| taskkill -f -im python
taskkill -f -im python.exe
|
关闭全部 uwsgi 进程
1
| ps aux|grep uwsgi|awk '{print $2}'|xargs kill -9
|
使用 uwsgi 启动 Django
如何在 Django 中任意安全获取 request
在 Django 中，request 包含了一次请求的全部信息。后端处理逻辑经常需要用到 request 中的信息。比如， DRF 框架中想要随时能够获取到 request，或者将一些参数全局传递。Django 第三方 App 中有一些工具可以满足要求，但它们并不是安全可靠的。意思是，如果 Django 启动时，使用了多线程或协程，在获取 request 时，可能会发生错误。这显然是不能接受的。下面是一个安全可靠的实现版本，让你在任意位置都能获取 request 对象。
1. 实现
utils/local.py 文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
| # -*- coding: utf-8 -*-
"""Thread-local/Greenlet-local objects
Thread-local/Greenlet-local objects support the management of
thread-local/greenlet-local data. If you have data that you want
to be local to a thread/greenlet, simply create a
thread-local/greenlet-local object and use its attributes:
>>> mydata = Local()
>>> mydata.number = 42
>>> mydata.number
42
>>> hasattr(mydata, 'number')
True
>>> hasattr(mydata, 'username')
False
Reference :
from threading import local
"""
try:
from greenlet import getcurrent as get_ident
except ImportError:
try:
from thread import get_ident
except ImportError:
from _thread import get_ident
__all__ = ["local", "Local"]
class Localbase(object):
__slots__ = ('__storage__', '__ident_func__')
def __new__(cls, *args, **kwargs):
self = object.__new__(cls, *args, **kwargs)
object.__setattr__(self, '__storage__', {})
object.__setattr__(self, '__ident_func__', get_ident)
return self
class Local(Localbase):
def __iter__(self):
ident = self.__ident_func__()
return iter(self.__storage__[ident].items())
def __release_local__(self):
self.__storage__.pop(self.__ident_func__(), None)
def __getattr__(self, name):
ident = self.__ident_func__()
try:
return self.__storage__[ident][name]
except KeyError:
raise AttributeError(name)
def __setattr__(self, name, value):
if name in ('__storage__', '__ident_func__'):
raise AttributeError(
"%r object attribute '%s' is read-only"
% (self.__class__.__name__, name))
ident = self.__ident_func__()
storage = self.__storage__
try:
storage[ident][name] = value
except KeyError:
storage[ident] = {name: value}
def __delattr__(self, name):
if name in ('__storage__', '__ident_func__'):
raise AttributeError(
"%r object attribute '%s' is read-only"
% (self.__class__.__name__, name))
ident = self.__ident_func__()
try:
del self.__storage__[ident][name]
if len(self.__storage__[ident]) == 0:
self.__release_local__()
except KeyError:
raise AttributeError(name)
local = Local()
if __name__ == '__main__':
def display(id):
# import time
local.id = id
for i in range(3):
print get_ident(), local.id, "\n"
# time.sleep(1)
def gree(id):
import gevent
t = []
for i in range(10):
t.append(gevent.spawn(display, "%s-%s" % (id, i)))
gevent.joinall(t)
# test one
# l1 = Local()
# l2 = Local()
# l.xxx = 1
# print l.xxx
# print l1.xxx
# print l2.xxx
# test two
# import gevent
# t = []
# for i in range(10):
# g = gevent.spawn(display, i)
# t.append(g)
# gevent.joinall(t)
# test three
import threading
t = []
for i in range(10):
t.append(threading.Thread(target=gree, args=(i,)))
[th.start() for th in t]
[th.join() for th in t]
|
utils/request_middlewares.py 文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
| # -*- coding: utf-8 -*-
from django.dispatch import Signal
from django.conf import settings
from utils.local import local
class AccessorSignal(Signal):
allowed_receiver = 'utils.request_middlewares.RequestProvider'
def __init__(self, providing_args=None):
Signal.__init__(self, providing_args)
def connect(self, receiver, sender=None, weak=True, dispatch_uid=None):
receiver_name = '.'.join(
[receiver.__class__.__module__, receiver.__class__.__name__]
)
if receiver_name != self.allowed_receiver:
raise Exception(
u"%s is not allowed to connect" % receiver_name)
if not self.receivers:
Signal.connect(self, receiver, sender, weak, dispatch_uid)
request_accessor = AccessorSignal()
class RequestProvider(object):
"""
@summary: request事件接收者
"""
def __init__(self):
request_accessor.connect(self)
def process_request(self, request):
"""
这里可以在 request 上添加自定义的一些数据、处理逻辑
"""
local.current_request = request
return None
def process_view(self, request, view_func, view_args, view_kwargs):
your_args = view_kwargs.get("your_args", "")
if not your_args:
your_args = (request.POST.get('your_args') or
request.GET.get('your_args')) or ""
request.your_args = your_args
def process_response(self, request, response):
if hasattr(local, 'current_request'):
assert request is local.current_request
del local.current_request
return response
def __call__(self, **kwargs):
if not hasattr(local, 'current_request'):
raise Exception(
u"get_request can't be called in a new thread.")
return local.current_request
def get_request():
if hasattr(local, 'current_request'):
return local.current_request
else:
raise Exception(u"get_request: current thread hasn't request.")
def get_x_request_id():
x_request_id = ''
http_request = get_request()
if hasattr(http_request, 'META'):
meta = http_request.META
x_request_id = (meta.get('HTTP_X_REQUEST_ID', '')
if isinstance(meta, dict) else '')
return x_request_id
|
2. 使用
1
2
3
4
| MIDDLEWARE_CLASSES = (
...
'utils.request_middlewares.RequestProvider',
... )
|
1
2
3
4
5
| from utils.request_middlewares import local
def my_function():
local.current_request
pass
|
Go mod 使用问题
1. 升级 Go 版本之后 go.sum 版本不匹配
执行命令 go build 报错。
错误提示：
missing go.sum entry for module providing package golang.org/x/time/rate; to add
解决办法：
在 go build 之前更新 go.sum，执行命令 go mod tidy
2. tls 错误
执行命令 go mod download 报错。
错误提示：
fatal: unable to access 'https://github.com/agiledragon/gomonkey/': GnuTLS recv error (-110): The TLS connection was non-properly terminated.
解决办法：
禁用证书校验执行命令，export GIT_SSL_NO_VERIFY=1 或 git config --global http.sslVerify false。
3. goproxy 不支持 latest
执行命令 go mod download 报错。
错误提示：
github.com/agiledragon/gomonkey: no matching versions for query "latest"
解决办法：
换其他 goproxy 代理。
NPM 常用命令
** 默认执行命名时，针对的是当前目录环境。如果加上 -g 参数，则表示针对的是全局生效。**
1
| npm config get registry
|
1
2
| npm config set registry http://registry.npm.taobao.org/
# npm config set registry https://registry.npmjs.org/
|
1
| npm install [package name] --registry=https://registry.npm.taobao.org
|
1
| npm config set proxy http://proxy.example.com:8080
|
1
| npm config delete proxy
|
1
| npm install [package name] -g
|
1
| npm install [package name] --save
|
1
| npm install [package name] --save-dev
|
1
| npm install [package name]@latest --save
|
1
| npm install [package name]@1.0.0 --save
|
- 指定安装大于等于@1.0.0 小于 2.2.0 版本
1
| npm install [package name]@">=2.0.0 <1.0.0" --save
|
1
| npm uninstall [package name] -g
|
1
| npm uninstall [package name]
|
Django Snippets
1. Admin 自动注册全部 Model 字段
admin.py
1
2
3
4
5
6
7
8
9
10
| # -*- coding: utf-8 -*-
import inspect
from django.contrib import admin
from . import models
for name, obj in inspect.getmembers(models):
try:
if inspect.isclass(obj):
admin.site.register(getattr(models, name))
except Exception as e:
pass
|
2. 获取全部 View Name
获取 Project 全部 View Name
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
| from django.conf import settings
from django.core.urlresolvers import RegexURLResolver, RegexURLPattern
root_urlconf = __import__(settings.ROOT_URLCONF)
all_urlpatterns = root_urlconf.urlpatterns
VIEW_NAMES = [] # maintain a global list
def get_all_view_names(urlpatterns):
global VIEW_NAMES
for pattern in urlpatterns:
if isinstance(pattern, RegexURLResolver):
get_all_view_names(pattern.url_patterns) # call this function recursively
elif isinstance(pattern, RegexURLPattern):
view_name = pattern.callback.func_name # get the view name
VIEW_NAMES.append(view_name) # add the view to the global list
return VIEW_NAMES
get_all_view_names(all_urlpatterns)
|
获取 App 全部 View Name
1
2
3
| from my_app.urls import urlpatterns as my_app_urlpatterns
my_app_views = get_all_view_names(my_app_urlpatterns)
|
3. Admin 中 ManyToMany Field 支持搜索
models.py
1
2
| class SomeModel(models.Model):
users = models.ManyToMany(User)
|
admin.py
1
2
| class SomeModelAdmin(admin.ModelAdmin):
filter_horizontal = ('users',)
|
4. Model 中 Choices 使用
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
| # -*- coding: utf-8 -*-
from enum import Enum
class ChoiceEnum(Enum):
@classmethod
def choices(cls):
return tuple((x.name, x.value) for x in cls)
@classmethod
def choices_name(cls):
return tuple(x.name for x in cls)
@classmethod
def choices_value(cls):
return tuple(x.value for x in cls)
@classmethod
def get_name(cls, value):
if value in cls.choices_value():
return cls.choices_name()[cls.choices_value().index(value)]
else:
return ''
@classmethod
def get_value(cls, name):
if name in cls.choices_name():
return cls.choices_value()[cls.choices_name().index(name)]
else:
return ''
class Colors(ChoiceEnum):
RED = 'red'
WHITE = 'white'
BLUE = 'blue'
|
models.py
1
2
| class Desk(models.Model):
color = models.CharField(max_length=32, choices=Colors.choices(), default=Colors.RED.name)
|
console 中
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| print Colors.choices()
(('BLUE', 'blue'), ('RED', 'red'), ('WHITE', 'white'))
print Colors.RED.value
red
print Colors.get_name(Colors.RED.value)
RED
print Colors.choices_value()
('blue', 'red', 'white')
print Colors.choices_name()
('BLUE', 'RED', 'WHITE')
|
5. 字符串与时间相互转换
1
2
3
| from datetime import datetime
print datetime.strptime('2017-09-02 17:41:20', '%Y-%m-%d %H:%M:%S')
datetime.datetime(2017, 9, 2, 17, 41, 20)
|
1
2
3
| from datetime import datetime
print datetime.now().strftime('%Y-%m-%d %H:%M:%S')
'2017-09-30 09:50:57'
|
6. 组装装饰器
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| def compose_decorators(*funs):
"""
按照顺序组装装饰器
@A
@B
@C
等价于
@compose_decorators(A, B，C)
"""
def deco(f):
for fun in reversed(funs):
f = fun(f)
return f
return deco
|
VS Code 无法下载 Go 插件的工具包
VS Code 安装 Go 插件之后，打开 Go 项目时，编辑器会提示安装工具包。
但是 golang.org 无法直接访问，我们需要将包手动下载到 GOPATH 目录，进行安装。
- 创建包目录
1
2
| cd $GOPATH
mkdir -p src/golang.org/x
|
- 下载包
1
2
3
| cd src/golang.org/x
git clone https://github.com/golang/tools.git
git clone https://github.com/golang/lint.git
|
- 安装包
1
2
| go get golang.org/x/tools/...
go get github.com/golangci/golangci-lint/cmd/golangci-lint@v1.27.0
|
