作者:MeshCloud脉时云公有云架构师王明立
引言
CDN分发的内容默认为公开资源,URL鉴权功能主要用于保护用户站点资源,防止资源被用户恶意下载盗用。
整体架构
请求具有签名 | 缓存命中 | 行为 |
否 | 否 | 转发到后端源站。 |
否 | 是 | 从缓存传送。 |
是 | 否 | 验证签名。如果有效,则转发到后端源站。 |
是 | 是 | 验证签名。如果有效,则从缓存传送。 |
技术简介
签名网址允许客户端临时访问不公开的资源,而无需获得额外授权。为了达到这个目的,系统将对请求的选定元素执行哈希处理,并使用您生成的强随机密钥进行加密签名。
当请求使用您提供的签名网址时,系统会认为请求有权接收请求的内容。当 Cloud CDN 收到一个针对某项已启用的服务发出的请求,而该请求具有错误的签名时,将拒绝该请求,并且永远不会将该请求发送至您的后端进行处理。
如何对网址签名
要对网址进行签名,您需要首先在后端服务或后端存储分区中创建一个或多个加密密钥。然后,您可以使用 Google Cloud CLI 或您自己的代码对网址进行哈希签名和加密。
签名网址的处理
在后端启用签名网址处理功能后,Cloud CDN 会对带有签名网址的请求进行特殊处理。具体而言,具有 Signature 查询参数的请求会被视为带有签名。收到此类请求后,Cloud CDN 将验证以下内容:
- HTTP 方法是 GET、HEAD、OPTIONS 或 TRACE。
- 将 Expires 参数设置为将来的某个时间。
- 请求的签名与使用指定密钥计算得出的签名匹配。
如果其中任何一项检查失败,系统都会提供 403 Forbidden 响应。其他情况下,请求会被代理到后端,或从缓存中传送相应的响应。 (OPTIONS 和 TRACE 请求始终直接经由代理发送到后端,不会从缓存传送。)特定基础网址(Expires 参数之前的部分)的所有有效签名请求将共享相同的缓存条目。签名请求和未签名请求的响应不会共享缓存条目。系统将缓存并传送响应,直到达到您设置的过期时间为止。
系统通常会使用 Cache-Control 标头将需要签名请求的内容标记为不可缓存。为了使此类对象与 Cloud CDN 兼容(并且无需在后端做出更改),Cloud CDN 将在响应具有有效签名网址的请求时替换 Cache-Control 标头。Cloud CDN 将这些内容视为可缓存的内容,并会使用 Cloud CDN 配置中设置的 max-age 参数。传送的响应仍然具有后端生成的 Cache-Control 标头。默认缓存1H,可以修改。
实施步骤
配置鉴权key及秘钥
需要在LB的后端选择url鉴权或者cookie鉴权,name即为鉴权key名称,可以使用自动创建的key或者手动输入的key(base64 编码的 128 位值),点击创建等待十分钟生效。
生成鉴权url
鉴权有两种方式URL鉴权和cookie鉴权,生成鉴权方式可以通过gcloud或者SDK,URL鉴权可以实现相同的url管理多个用户,到期后所有用户将不能访问,cookie鉴权可以实现管理单个用户使用单个或多个资源,不同的场景使用不同的鉴权方式。
- gcloud 方式
gcloud compute sign-url "URL" --key-name KEY_NAME --key-file KEY_FILE_NAME --expires-in TIME_UNTIL_EXPIRATION
URL 必须是包含路径某个组成部分的有效网址。例如,http://example.com 无效,不过 https://example.com/ 和 https://example.com/whatever 均为有效网址。
Key file 把第一步配置的key填写近一个文件里,并对应的文件名
–expires-in 过期时间 例如5m、1d
生成示例
gcloud 可以生成url鉴权地址,无法生成cookie鉴权地址,建议使用python
- Python 代码生成
#!/usr/bin/env python
#
# Copyright 2017 Google, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""This application demonstrates how to perform operations on data (content)
when using Google Cloud CDN (Content Delivery Network).
For more information, see the README.md under /cdn and the documentation
at https://cloud.google.com/cdn/docs.
"""
import argparse
import base64
import datetime
import hashlib
import hmac
from six.moves import urllib
# [START sign_url]
def sign_url(url, key_name, base64_key, expiration_time):
"""Gets the Signed URL string for the specified URL and configuration.
Args:
url: URL to sign as a string.
key_name: name of the signing key as a string.
base64_key: signing key as a base64 encoded string.
expiration_time: expiration time as a UTC datetime object.
Returns:
Returns the Signed URL appended with the query parameters based on the
specified configuration.
"""
stripped_url = url.strip()
parsed_url = urllib.parse.urlsplit(stripped_url)
query_params = urllib.parse.parse_qs(
parsed_url.query, keep_blank_values=True)
epoch = datetime.datetime.utcfromtimestamp(0)
expiration_timestamp = int((expiration_time - epoch).total_seconds())
decoded_key = base64.urlsafe_b64decode(base64_key)
url_pattern = u'{url}{separator}Expires={expires}&KeyName={key_name}'
url_to_sign = url_pattern.format(
url=stripped_url,
separator='&' if query_params else '?',
expires=expiration_timestamp,
key_name=key_name)
digest = hmac.new(
decoded_key, url_to_sign.encode('utf-8'), hashlib.sha1).digest()
signature = base64.urlsafe_b64encode(digest).decode('utf-8')
signed_url = u'{url}&Signature={signature}'.format(
url=url_to_sign, signature=signature)
print(signed_url)
def sign_url_prefix(url, url_prefix, key_name, base64_key, expiration_time):
"""Gets the Signed URL string for the specified URL prefix and configuration.
Args:
url: URL of request.
url_prefix: URL prefix to sign as a string.
key_name: name of the signing key as a string.
base64_key: signing key as a base64 encoded string.
expiration_time: expiration time as a UTC datetime object.
Returns:
Returns the Signed URL appended with the query parameters based on the
specified URL prefix and configuration.
"""
stripped_url = url.strip()
parsed_url = urllib.parse.urlsplit(stripped_url)
query_params = urllib.parse.parse_qs(
parsed_url.query, keep_blank_values=True)
encoded_url_prefix = base64.urlsafe_b64encode(
url_prefix.strip().encode('utf-8')).decode('utf-8')
epoch = datetime.datetime.utcfromtimestamp(0)
expiration_timestamp = int((expiration_time - epoch).total_seconds())
decoded_key = base64.urlsafe_b64decode(base64_key)
policy_pattern = u'URLPrefix={encoded_url_prefix}&Expires={expires}&KeyName={key_name}'
policy = policy_pattern.format(
encoded_url_prefix=encoded_url_prefix,
expires=expiration_timestamp,
key_name=key_name)
digest = hmac.new(
decoded_key, policy.encode('utf-8'), hashlib.sha1).digest()
signature = base64.urlsafe_b64encode(digest).decode('utf-8')
signed_url = u'{url}{separator}{policy}&Signature={signature}'.format(
url=stripped_url,
separator='&' if query_params else '?',
policy=policy,
signature=signature)
print(signed_url)
# [END sign_url]
# [START cdn_sign_cookie]
def sign_cookie(url_prefix, key_name, base64_key, expiration_time):
"""Gets the Signed cookie value for the specified URL prefix and configuration.
Args:
url_prefix: URL prefix to sign as a string.
key_name: name of the signing key as a string.
base64_key: signing key as a base64 encoded string.
expiration_time: expiration time as a UTC datetime object.
Returns:
Returns the Cloud-CDN-Cookie value based on the specified configuration.
"""
encoded_url_prefix = base64.urlsafe_b64encode(
url_prefix.strip().encode('utf-8')).decode('utf-8')
epoch = datetime.datetime.utcfromtimestamp(0)
expiration_timestamp = int((expiration_time - epoch).total_seconds())
decoded_key = base64.urlsafe_b64decode(base64_key)
policy_pattern = u'URLPrefix={encoded_url_prefix}:Expires={expires}:KeyName={key_name}'
policy = policy_pattern.format(
encoded_url_prefix=encoded_url_prefix,
expires=expiration_timestamp,
key_name=key_name)
digest = hmac.new(
decoded_key, policy.encode('utf-8'), hashlib.sha1).digest()
signature = base64.urlsafe_b64encode(digest).decode('utf-8')
signed_policy = u'Cloud-CDN-Cookie={policy}:Signature={signature}'.format(
policy=policy, signature=signature)
print(signed_policy)
# [END cdn_sign_cookie]
if __name__ == '__main__':
parser = argparse.ArgumentParser(
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
subparsers = parser.add_subparsers(dest='command')
sign_url_parser = subparsers.add_parser(
'sign-url',
help="Sign a URL to grant temporary authorized access.")
sign_url_parser.add_argument(
'url', help='The URL to sign.')
sign_url_parser.add_argument(
'key_name',
help='Key name for the signing key.')
sign_url_parser.add_argument(
'base64_key',
help='The base64 encoded signing key.')
sign_url_parser.add_argument(
'expiration_time',
type=lambda d: datetime.datetime.utcfromtimestamp(float(d)),
help='Expiration time expessed as seconds since the epoch.')
sign_url_prefix_parser = subparsers.add_parser(
'sign-url-prefix',
help="Sign a URL prefix to grant temporary authorized access.")
sign_url_prefix_parser.add_argument(
'url', help='The request URL.')
sign_url_prefix_parser.add_argument(
'url_prefix', help='The URL prefix to sign.')
sign_url_prefix_parser.add_argument(
'key_name',
help='Key name for the signing key.')
sign_url_prefix_parser.add_argument(
'base64_key',
help='The base64 encoded signing key.')
sign_url_prefix_parser.add_argument(
'expiration_time',
type=lambda d: datetime.datetime.utcfromtimestamp(float(d)),
help='Expiration time expessed as seconds since the epoch.')
sign_cookie_parser = subparsers.add_parser(
'sign-cookie',
help="Generate a signed cookie to grant temporary authorized access.")
sign_cookie_parser.add_argument(
'url_prefix', help='The URL prefix to sign.')
sign_cookie_parser.add_argument(
'key_name',
help='Key name for the signing key.')
sign_cookie_parser.add_argument(
'base64_key',
help='The base64 encoded signing key.')
sign_cookie_parser.add_argument(
'expiration_time',
type=lambda d: datetime.datetime.utcfromtimestamp(float(d)),
help='Expiration time expressed as seconds since the epoch.')
args = parser.parse_args()
if args.command == 'sign-url':
sign_url(
args.url, args.key_name, args.base64_key, args.expiration_time)
elif args.command == 'sign-url-prefix':
sign_url_prefix(
args.url, args.url_prefix, args.key_name, args.base64_key, args.expiration_time)
elif args.command == 'sign-cookie':
sign_cookie(
args.url_prefix, args.key_name, args.base64_key, args.expiration_time)
生成示例
测试验证
url鉴权
鉴权通过返回200,过期时间5分钟,还有注意我们上面提到的坑,不带鉴权参数是放行的
cookie鉴权
鉴权通过返回200,过期时间5分钟
常见问题
带参数的url是否能作鉴权?
URLPrefix 会对架构(http:// 或 https://)、FQDN 和可选路径进行编码。您可以选择是否使用以 / 结尾的路径,但我们建议您使用。前缀不应包含查询参数或者 ? 或 # 这样的片段。
例如,https://media.example.com/videos 会将请求匹配到以下两项:
https://media.example.com/videos?video_id=138183&user_id=138138
https://media.example.com/videos/137138595?quality=low
修改鉴权url缓存时间方法?
设置后端服务
gcloud compute backend-services update BACKEND_NAME
–signed-url-cache-max-age MAX_AGE
后端存储分区
gcloud compute backend-buckets update BACKEND_NAME
–signed-url-cache-max-age MAX_AGE