https://www.chenshaowen.com/en/tags/django/Recent content in Django on Shaowen Chen's WebsiteHugo -- gohugo.ioen©2016 - {year}, All Rights Reserved.Wed, 20 Mar 2019 00:00:00 +0000weeklyhttps://www.chenshaowen.com/en/blog/database-query-optimization-of-django-performance.htmlWed, 20 Mar 2019 00:00:00 +0000Wed, 20 Mar 2019 00:00:00 +0000https://www.chenshaowen.com/en/blog/database-query-optimization-of-django-performance.htmlThis article mainly offers optimization advice on Django fields and queries, and also introduces a performance analysis tool called Django-silk. I hope it helps you develop high-performance Django projects. 1. DBA’s Advice 1.1 Table Field Design Avoid null values; null values are hard to optimize queries around and take up extra index space Prefer INT over BIGINT, and describe fields as accurately as possible Use enums or integers instead of string types Use TIMESTAMP instead of DATETIME Do not put more than 20 fields in a single table Store IPs as integers 1.微信公众号DatabaseOptimizationPerformanceDjangoPythonR&Dhttps://www.chenshaowen.com/en/blog/sub-db-and-table-of-django-performance.htmlTue, 19 Mar 2019 00:00:00 +0000Tue, 19 Mar 2019 00:00:00 +0000https://www.chenshaowen.com/en/blog/sub-db-and-table-of-django-performance.html1. The Problem We Hit The frontend requests are heavy, concurrency is high, and access is slow. The bottlenecks show up mainly as: Large single tables Large single databases Slow network IO Slow disk IO Optimizing network and disk IO mainly relies on hardware upgrades. In theory, a database imposes no limit on the size of a single database or a single table, but an oversized single database or table means more requests land on a single machine, putting pressure on IO.微信公众号DjangoDatabaseShardingPerformanceDatabase ShardingTable Shardinghttps://www.chenshaowen.com/en/blog/how-to-use-jenkins-docker-gitlab-to-build-django-automated-deployment-process.htmlSun, 25 Nov 2018 00:00:00 +0000Sun, 25 Nov 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-use-jenkins-docker-gitlab-to-build-django-automated-deployment-process.htmlOne of the illusions that programmers at big companies easily fall into is mistaking platform capabilities for their own abilities. In a large team, we should not focus only on our own little patch of ground; we need to understand every part of the platform. On one hand, this helps us make better use of the platform’s features; on the other, it helps our own technical growth.微信公众号DjangoGitLabJenkinsDockerDevOpsCICDR&DOperationshttps://www.chenshaowen.com/en/blog/api-of-viewset-and-serializer-in-rest-framework.htmlTue, 20 Nov 2018 00:00:00 +0000Tue, 20 Nov 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/api-of-viewset-and-serializer-in-rest-framework.html1. Serializer 1.1 Data Validation When deserializing data, the validity of the data needs to be checked. At this point you can call is_valid() for validation, and if a validation error occurs, you can get the error message from the .errors attribute. For example: 1 2 3 4 serializer.is_valid() # False serializer.微信公众号DjangoPythonAPIInterfaceR&Dhttps://www.chenshaowen.com/en/blog/viewset-and-serializer-in-rest-framework.htmlSun, 18 Nov 2018 00:00:00 +0000Sun, 18 Nov 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/viewset-and-serializer-in-rest-framework.html1. View Class in Django First, recall how Django handles a request. After receiving a request, Django creates a handler of type WSGIHandler, and the handler controls the entire processing flow. So how are the request URL and the View associated with each other? Django first loads the URLconf according to the ROOT_URLCONF setting, then matches the URLpatterns in the URLconf one by one in order, stopping as soon as a match is found.微信公众号DjangoPythonAPIFrameworkInterfaceR&Dhttps://www.chenshaowen.com/en/blog/django-full-stack-optimization-guide.htmlThu, 15 Nov 2018 00:00:00 +0000Thu, 15 Nov 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/django-full-stack-optimization-guide.htmlAs the data volume exploded, the system responded very slowly. We carried out a series of optimizations on the application, and the system’s response time improved by an order of magnitude. Overall, optimizations in file compression and faster network access gave a noticeable boost to frontend performance, while optimizations in stored procedures, caching, and logic code gave a noticeable boost to backend performance.微信公众号DjangoPythonOptimizationFrontendBackendGuideR&Dhttps://www.chenshaowen.com/en/blog/solving-the-problem-of-celery-processe-large-file-failure.htmlWed, 24 Oct 2018 00:00:00 +0000Wed, 24 Oct 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/solving-the-problem-of-celery-processe-large-file-failure.html1. A Small Requirement We run into small requirements all the time, but they are not always simple to implement. Here is a simple file upload requirement, broken into the following steps: The user uploads a large file on the page The large file is temporarily stored in the internal微信公众号CeleryPythonDjangoFailureTroubleshootingLarge FilesR&Dhttps://www.chenshaowen.com/en/blog/python-and-django-development-practices.htmlFri, 19 Oct 2018 00:00:00 +0000Fri, 19 Oct 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/python-and-django-development-practices.htmlCompiled from the “Development Tips” series, collecting common problems and solutions in Python and Django development. 1. Sorting Python Iterables Iterable objects fall into 3 main categories: All sequence types, such as list, str, and tuple Some non-sequence types, such as dict and file Objects that contain an __iter__() or微信公众号PythonDjangoR&DLearningBackendhttps://www.chenshaowen.com/en/blog/how-to-use-django-signal-decoupling-module.htmlFri, 12 Oct 2018 00:00:00 +0000Fri, 12 Oct 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-use-django-signal-decoupling-module.htmlRecently I was responsible for developing a backend-heavy application. The data flow in this application is complex, and the processing logic has piled up in a redundant mess. The project’s tech stack is Django + Vuejs. The frontend is bundled with Webpack, managed in modules, and mainly displays data. The backend involves many modules, many processing rules, and many data tables, so every time I had to modify an earlier feature I spent a lot of time reviewing the code.微信公众号DjangoPythonDesignDecouplingR&Dhttps://www.chenshaowen.com/en/blog/how-to-secure-get-request-in-django.htmlTue, 26 Jun 2018 00:00:00 +0000Tue, 26 Jun 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-secure-get-request-in-django.htmlIn Django, request holds all the information for a single request. Backend processing logic frequently needs information from request. For example, the DRF framework may want to be able to obtain request at any time, or pass some parameters globally. There are third-party Django apps with tools that can satisfy微信公众号DjangoPythonR&DTipsLearningDRFThread Safetyhttps://www.chenshaowen.com/en/blog/django-model-and-er-diagram.htmlFri, 02 Mar 2018 00:00:00 +0000Fri, 02 Mar 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/django-model-and-er-diagram.htmlAs requirements are gradually confirmed, the system’s DB model is gradually confirmed as well. Sometimes you use a tool like MySQL Workbench to draw an E-R diagram and then generate MySQL database tables, and at that point you need to reverse-generate Django Models; other times you write the Django Models directly, but then you need to look at the E-R diagram.微信公众号DjangoPythonDatabaseDesignModelR&Dhttps://www.chenshaowen.com/en/blog/development-specification-1-of-django.htmlSat, 27 Jan 2018 00:00:00 +0000Sat, 27 Jan 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/development-specification-1-of-django.htmlThis article mainly lays out a few things to watch out for during Django development. A consistent coding style and sound design principles help a project’s development and maintenance, and are worth developers studying and discussing continuously. 1. Encoding Declaration When the Python interpreter executes code, it needs to be微信公众号DjangoPythonConventionsR&DLearningBest PracticesCode StyleArchitecturehttps://www.chenshaowen.com/en/blog/stateless-service.htmlWed, 03 Jan 2018 17:22:45 +0000Wed, 03 Jan 2018 17:22:45 +0000https://www.chenshaowen.com/en/blog/stateless-service.html1. Problem Description Background: a SaaS application developed with Django, offering document services externally. Its search feature is implemented through Django Haystack. Problem: the search feature is sometimes available and sometimes not. After repeated testing, the available and unavailable states were found to alternate, each occurring with a probability of微信公众号DjangoServicesPaaSSaaShttps://www.chenshaowen.com/en/blog/api-interface-specification.htmlFri, 01 Dec 2017 00:00:00 +0000Fri, 01 Dec 2017 00:00:00 +0000https://www.chenshaowen.com/en/blog/api-interface-specification.htmlIn the process of Web application development, backend developers need to deliver API interfaces frequently, and frontend developers need to call API interfaces frequently. To lower communication costs and prevent possible security risks, and following the principle of convention over configuration, it is necessary to standardize the API specification. Restful微信公众号DjangoInterfaceAPIR&DSpecificationhttps://www.chenshaowen.com/en/blog/error-code-design-and-unified-processing-in-django.htmlMon, 30 Oct 2017 21:50:33 +0000Mon, 30 Oct 2017 21:50:33 +0000https://www.chenshaowen.com/en/blog/error-code-design-and-unified-processing-in-django.htmlI currently use Django for SaaS development, and I develop and maintain several SaaS applications at the same time. Many SaaS applications have agreed-upon error codes, some used to handle login state, others to mark business logic status. For a feature that is so strongly shared across projects, it is微信公众号DjangoPythonError CodesMiddlewareR&DLearningAPIBest Practiceshttps://www.chenshaowen.com/en/blog/control-of-static-file-version-in-django.htmlFri, 06 Oct 2017 09:58:43 +0000Fri, 06 Oct 2017 09:58:43 +0000https://www.chenshaowen.com/en/blog/control-of-static-file-version-in-django.htmlTo respond quickly to user needs and meet the requirements of marketing campaigns, internet products usually have a very high release frequency. Adopting agile development shortens the delivery cycle and speeds up product iteration, but it also brings challenges to a project’s file management. Frontend engineering faces users directly, bears微信公众号DjangoFrontendJavaScriptFilesR&DVersionhttps://www.chenshaowen.com/en/blog/build-django-development-environment-using-vagrant.htmlMon, 25 Sep 2017 21:56:05 +0000Mon, 25 Sep 2017 21:56:05 +0000https://www.chenshaowen.com/en/blog/build-django-development-environment-using-vagrant.htmlDocker solves the problem of environment isolation on the same machine and improves the efficiency of operations and deployment. Vagrant gives developers a unified, near-fully-isolated environment for development and testing. This article mainly discusses how to build a Django development environment with Vagrant. Versions: VirtualBox 5.0, Vagrant 1.8. 1. Basic微信公众号DjangoPythonVagrantToolsDevelopmentR&DOperationshttps://www.chenshaowen.com/en/blog/how-to-package-django-development-environments-using-docker-6.htmlSun, 24 Sep 2017 18:03:22 +0000Sun, 24 Sep 2017 18:03:22 +0000https://www.chenshaowen.com/en/blog/how-to-package-django-development-environments-using-docker-6.html1. Directory Structure The django-devops-uwsgi directory holds the image build files that package uWSGI and Django. It installs the necessary packages such as uWSGI, pip, and virtualenv into the image. When a container is created from the image, the start.sh script runs: it creates the Python virtual runtime environment, installs the dependency packages from Django’s requirements.微信公众号DockerDjangoPythonServicesToolsR&DOperationshttps://www.chenshaowen.com/en/blog/how-to-package-django-development-environments-using-docker-5.htmlSun, 24 Sep 2017 14:34:35 +0000Sun, 24 Sep 2017 14:34:35 +0000https://www.chenshaowen.com/en/blog/how-to-package-django-development-environments-using-docker-5.html1. Directory Structure 1 2 3 4 5 6 7 8 9 10 11 12 ├── django-devops-nginx │ ├── Dockerfile │ └── nginx.repo ├── django-devops-compose │ ├── docker-compose.yml │ ├── www │ ├── conf │ │ ├── nginx.conf │ ├── log │ │ ├── nginx │ │ │ ├── access.log │ │ │ ├── error.微信公众号DockerServicesToolsDjangoNginxPythonR&DOperationshttps://www.chenshaowen.com/en/blog/how-to-package-django-development-environments-using-docker-3.htmlSat, 23 Sep 2017 19:56:03 +0000Sat, 23 Sep 2017 19:56:03 +0000https://www.chenshaowen.com/en/blog/how-to-package-django-development-environments-using-docker-3.html1. Basic Concepts Docker Compose is a Docker tool for defining and running complex applications. With Docker Compose, you can define a multi-container application in a single file, then use a single command to start your application and finish all the preparation work. Docker Compose positions itself as ‘defining and running complex applications with Docker’.微信公众号DockerServicesToolsDjangoPythonR&DOperationshttps://www.chenshaowen.com/en/blog/how-to-package-django-development-environments-using-docker-2.htmlFri, 22 Sep 2017 19:52:27 +0000Fri, 22 Sep 2017 19:52:27 +0000https://www.chenshaowen.com/en/blog/how-to-package-django-development-environments-using-docker-2.html1. Basic Concepts A Dockerfile is a collection of instructions for building a Docker image. Docker reads Dockerfile instructions to build an image automatically. A Dockerfile is similar to a Makefile: both are text files that organize all the instructions in the order the image is built. The command to build a Docker image:微信公众号DockerDjangoPythonServicesToolsR&DOperationshttps://www.chenshaowen.com/en/blog/django-application-performance-test-based-on-docker.htmlThu, 07 Sep 2017 19:08:09 +0000Thu, 07 Sep 2017 19:08:09 +0000https://www.chenshaowen.com/en/blog/django-application-performance-test-based-on-docker.html1. Basic Website Performance Metrics Concurrency: the number of sessions the server handles per unit of time Throughput: the number of requests the server handles per unit of time Response time: the total elapsed time from when the user sends a request to when the response is fully received, made up of network transmission time, service processing time, and other parts Average response time: the average of the response times of all requests 2.微信公众号TestingDemoDockerDjangohttps://www.chenshaowen.com/en/blog/django-forms-function.htmlFri, 21 Jul 2017 21:29:40 +0000Fri, 21 Jul 2017 21:29:40 +0000https://www.chenshaowen.com/en/blog/django-forms-function.html1. Automatically Generating HTML Form Elements A Widget is the tool used to render an HTML element. Specifying a widget 1 2 3 4 5 6 from django import forms class CommentForm(forms.Form): name = forms.CharField() url = forms.URLField() comment = forms.CharField(widget=forms.Textarea) Output of CommentForm().as_table() 1 2 3 4 5 6微信公众号DjangoPythonSecurityR&Dhttps://www.chenshaowen.com/en/blog/sql-code-about-django-orm.htmlFri, 21 Jul 2017 13:21:08 +0000Fri, 21 Jul 2017 13:21:08 +0000https://www.chenshaowen.com/en/blog/sql-code-about-django-orm.html1. Basic Concepts ORM: Object Relational Mapping. Its job is to map between a relational database and objects. No complex SQL statements are needed — operating on data is as simple as operating on objects. QuerySet: the list of objects for a given model. A QuerySet lets you read data微信公众号DjangoPythonDataSQLR&Dhttps://www.chenshaowen.com/en/blog/haystack-full-text-search.htmlFri, 21 Jul 2017 13:00:03 +0000Fri, 21 Jul 2017 13:00:03 +0000https://www.chenshaowen.com/en/blog/haystack-full-text-search.htmlA quick word on the project requirements: the team needed to publish documentation externally. The documents are written in Markdown and need to be published as HTML. At first we used an Nginx + Jekyll solution. As the documentation grew, the document system developed a strong need for search. I微信公众号DjangoPythonDataDesignSearchR&Dhttps://www.chenshaowen.com/en/blog/signal-of-django.htmlThu, 13 Jul 2017 13:06:10 +0000Thu, 13 Jul 2017 13:06:10 +0000https://www.chenshaowen.com/en/blog/signal-of-django.html1. Basic Concepts Django ships with a signal dispatcher built in. Signals help decouple program modules. When an event occurs elsewhere in the application, a designated function is notified. Signals let certain senders notify a group of receivers that some action has taken place. 2. Using Signals 2.1 Declaring a微信公众号DjangoBackendPythonR&DLearningDecouplingEventshttps://www.chenshaowen.com/en/blog/class-based-view-of-django.htmlTue, 11 Jul 2017 16:36:56 +0000Tue, 11 Jul 2017 16:36:56 +0000https://www.chenshaowen.com/en/blog/class-based-view-of-django.htmlDjango has two kinds of views: function-based views and class-based views. The role of a view is mainly to fill in logic and return a response body. Function-based views are hard to extend and have a low rate of code reuse. Class-based views, on the other hand, can use inheritance微信公众号DjangoBackendPythonClass-Based ViewsR&Dhttps://www.chenshaowen.com/en/blog/permissions-of-django-rest-framework.htmlFri, 07 Jul 2017 06:47:19 +0000Fri, 07 Jul 2017 06:47:19 +0000https://www.chenshaowen.com/en/blog/permissions-of-django-rest-framework.html1. Permission Management in DRF Permission management in Django REST Framework consists of two parts. One is Authentication. It specifies how the user is authenticated, obtaining request.user. The other is Permissions. It performs permission control over Django resources and user categories. 1.1 Authentication Methods The relevant source is in the rest_framework/authentication.微信公众号DjangoPythonBackendSecurityAPIPermissionsR&Dhttps://www.chenshaowen.com/en/blog/django-template-inherit.htmlMon, 03 Jul 2017 19:17:11 +0000Mon, 03 Jul 2017 19:17:11 +0000https://www.chenshaowen.com/en/blog/django-template-inherit.html1. The Scenario In a project, elements such as the header and footer are often reused. To avoid rewriting these elements on every page, and so that a change does not require editing each page individually, the common parts need to be extracted — that is Django template inheritance. 2.微信公众号DjangoPythonTemplatesR&Dhttps://www.chenshaowen.com/en/blog/django-decorator.htmlThu, 22 Jun 2017 23:55:59 +0000Thu, 22 Jun 2017 23:55:59 +0000https://www.chenshaowen.com/en/blog/django-decorator.htmlWhen developing with a separated front end and back end, some of the API endpoints provided to the front end use GET requests and some use POST requests. To keep the back end from throwing an error while reading values from request in views.py, you had to check the request微信公众号DjangoDemoDecoratorsPythonhttps://www.chenshaowen.com/en/blog/snippets-of-django.htmlThu, 22 Jun 2017 23:52:06 +0000Thu, 22 Jun 2017 23:52:06 +0000https://www.chenshaowen.com/en/blog/snippets-of-django.html1. Auto-Register All Models in Admin admin.py 1 2 3 4 5 6 7 8 9 10 # -*- coding: utf-8 -*- import inspect from django.contrib import admin from . import models for name, obj in inspect.getmembers(models): try: if inspect.isclass(obj): admin.site.register(getattr(models, name)) except Exception as e: pass 2. Get All View Names Get all view names of the project微信公众号DjangoPythonR&DLearningTipshttps://www.chenshaowen.com/en/blog/101-of-django-rest-framework.htmlThu, 22 Jun 2017 23:50:15 +0000Thu, 22 Jun 2017 23:50:15 +0000https://www.chenshaowen.com/en/blog/101-of-django-rest-framework.htmlIn the SaaS development I work on, there is a fairly high demand for development efficiency. From project initiation, prototype design and evaluation, requirements confirmation, frontend design, and backend development through to final acceptance, we complete one iteration in just a few weeks. Guided by agile development, we began rolling微信公众号DataBackendInterfaceDjangoPythonAPIR&Dhttps://www.chenshaowen.com/en/blog/practice-and-automation-generation-of-apidoc.htmlWed, 14 Jun 2017 17:22:45 +0000Wed, 14 Jun 2017 17:22:45 +0000https://www.chenshaowen.com/en/blog/practice-and-automation-generation-of-apidoc.htmlIn a frontend/backend separated architecture, API documentation changes hands frequently. When third-party interfaces are involved, API and documentation changes can be even faster in multi-party collaboration scenarios. To make maintaining the API and handing over documentation easier, here is a documentation generation tool worth recommending — apidoc. 1. Introduction to微信公众号DjangoInterfaceAPIDocumentationR&Dhttps://www.chenshaowen.com/en/blog/restful-api-of-using-django-tastypie.htmlTue, 13 Jun 2017 21:31:13 +0000Tue, 13 Jun 2017 21:31:13 +0000https://www.chenshaowen.com/en/blog/restful-api-of-using-django-tastypie.html1. Restful REST is short for Representational State Transfer, meaning “representation state transfer”. Restful is a development philosophy and a software architecture style for the World Wide Web. 1.1 Features of Restful Abstract resources Images, text, songs, and videos are all resource entities; on the network they are abstracted as微信公众号DjangoPythonInterfaceAPITastypieR&Dhttps://www.chenshaowen.com/en/blog/how-to-export-excel-quickly-in-web-development.htmlThu, 18 May 2017 17:51:05 +0000Thu, 18 May 2017 17:51:05 +0000https://www.chenshaowen.com/en/blog/how-to-export-excel-quickly-in-web-development.htmlIn web development you often run into the need to export data. This post mainly covers how to quickly export data and save it as an Excel file. 1. Frontend In web development, formatted data is usually displayed as a table. Below is an employee salary table; we will use微信公众号DjangoFrontendDataDemohttps://www.chenshaowen.com/en/blog/manage-command-in-django.htmlMon, 15 May 2017 21:14:48 +0000Mon, 15 May 2017 21:14:48 +0000https://www.chenshaowen.com/en/blog/manage-command-in-django.htmlEvery Django project automatically generates a manage.py file. manage.py is a simple wrapper around django-admin; its job is to put the Django Project into the sys.path directory and set the DJANGO_SETTINGS_MODULE environment variable to the current Project’s settings.py file. In other words, the difference between django-admin and manage is that微信公众号DjangoPythonToolsCommandR&Dhttps://www.chenshaowen.com/en/blog/how-to-build-reusable-django-app.htmlFri, 05 May 2017 21:43:47 +0000Fri, 05 May 2017 21:43:47 +0000https://www.chenshaowen.com/en/blog/how-to-build-reusable-django-app.htmlThe team I work on is responsible for SaaS development and carries almost all of the center’s SaaS needs. Among these there are long-term maintenance projects and short-term crunch demo projects, and everyone wears several hats. Of course, the development platform is also open to other people to use, and the whole platform has hundreds if not thousands of SaaS applications.微信公众号DjangoPythonManagementR&DArchitecturehttps://www.chenshaowen.com/en/blog/django-permissions-control.htmlSun, 22 Jan 2017 09:23:45 +0000Sun, 22 Jan 2017 09:23:45 +0000https://www.chenshaowen.com/en/blog/django-permissions-control.html1. Django Built-in Permission Management 1.1 Permission Categories Permission Used to define User A’s permission on Task. User If User A has permission on Model B, then User A has the corresponding permission on all instances in Model B. The user_permission field of the User object is used to manage微信公众号DjangoDevelopmentPermissionsPythonR&DLearningSecurityRBACAuthenticationhttps://www.chenshaowen.com/en/blog/several-methods-of-django-data-migration.htmlTue, 17 Jan 2017 23:29:15 +0000Tue, 17 Jan 2017 23:29:15 +0000https://www.chenshaowen.com/en/blog/several-methods-of-django-data-migration.html1. Background In the course of Web development, migrating data between multiple environments (local, testing, and production) comes up often. This article mainly discusses the possible data migration paths involved in Django development, and seeks feasible methods for them. 2. Scenarios There are four data migration targets in total: the微信公众号DjangoPythonDataMigrationR&D